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Abstract 

We analyse the non-perturbative superpotential due to M5-brane instantons in F-theory 
compactifications on Calabi-Yau fourfolds. The M5 partition function is obtained via holo- 
morphic factorisation by explicitly performing the sum over chiral 3- form fluxes. Compari- 
son with the partition function of fluxed Euclidean D3-brane instantons in Type IIB orien- 
tifolds allows us to flx the spin structure on the intermediate Jacobian of the M5-instanton. 
We furthermore analyse the contribution of the M5-instanton to the superpotential in the 
presence of G4 gauge flux, where the superpotential is dressed with matter fields. We ex- 
plicitly evaluate the pullback of G4 onto the M5-brane as a measure for the presence of 
charged instanton zero modes. This accounts for the M5 charge both under massless ?7(l)s, 
if present, and under what corresponds in Type II language to geometrically massive C/(l)s. 
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1 Introduction 



Instanton effects due to Euclidean branes have received widespread interest both in Type II 
string theory and in F/M-theory compactifications. Part of their importance is owed to the fact 
that they generate perturbatively absent terms in the superpotential of the four-dimensional 
effective action. Examples are non-perturbative F-terms involving the Kahler moduli in F/M- 
theory and Type IIB orientifolds [1], which are the backbone of recent approaches to moduli 
stabilisation in the spirit of [2], or non-perturbative couplings in the charged open string sector 
[SHS], which play an important role for phenomenology. 

The present paper is concerned with instanton effects due to Euclidean M5-branes in F- 
theory compactifications on a Calabi-Yau fourfold. One of our aims is to investigate when such 
an instanton contributes to a superpotential term involving only the geometric string moduli, 
and when it has to involve matter fields charged under gauge groups appearing on the 7-branes 
of the theory. To appreciate some of the problems that occur, let us recall what makes the study 
of instanton effects in perturbative Type II compactifications tractable and to what extent these 
properties carry over to F /M-theory. For a review of these techniques and the original literature 
see e.g. [Zl[8]. 

In Type II string theories, D-brane instanton effects can be studied rather explicitly by 
quantising the excitations of open strings ending on the Euclidean D-brane. This allows for a 
detailed understanding in particular of the zero mode spectrum of the instanton, which is crucial 
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in determining which superpotential terms the instanton can contribute to or if it contributes 
at all. 

The instanton moduli and their associated zero modes can be treated with techniques very 
similar to those used in the study of spacetime-filling D-branes. The zero modes arising from 
open strings with both ends on the instanton are called neutral zero modes, as they are un- 
charged under the gauge groups of stacks of spacetime-filling D-branes that may be present. 
In addition, charged zero modes can appear at the intersection of the instanton with a stack 
of spacetime-filling D-branes. The charged zero mode structure of an instanton can again be 
studied by explicitly quantising the open strings stretching between instanton and D-brane 
stack, taking into account the appearance of gauge flux, if present, in the relevant boundary 
conditions. Instead of quantizing the charged open string sector between the instanton and the 
spacetime-filling branes, one can look for the appearance of chiral zero modes by considering 
the transformation properties of the instanton action under the U{1) gauge symmetries on the 
D-brane stack. These U{1) selection rules imply that an instanton whose classical action carries 
a certain charge q can contribute at most to terms in the low-energy action involving suitable 
open string fields (p of charge —q. 

Of particular interest in this paper are Euclidean D3-brane instantons (short E3-instantons) 
in Type IIB orientifolds. As stressed in [9], the computation of the associated superpotential 
involves a sum over the set of consistent fluxes on the instanton worldvolume. Aspects of E3- 
instanton flux had previously been discussed in |10) . Instanton flux has important consequences 
for the spectrum of charged zero modes because the net chirality of these charged zero modes 
depends on the gauge flux both on the D-brane stack and on the instanton [S]. Among other 
things, this alleviates the tension between a chiral matter spectrum and moduli stabilisation by 
D-brane instantons, which was first pointed out in [11]. Instanton fluxes can also lift zero modes 
due to neutral instanton moduli [12] . with great impact on the structure of the superpotential. 

In F-theory compactifications, the phenomenologically important instanton effects men- 
tioned above are comparatively under much less control than in the perturbative Type IIB 
setting. F-theory can be defined as M-theory compactified on an elliptically fibered fourfold in 
the limit of vanishing fiber volume. The uplift of the E3-instantons of Type IIB discussed above 
is then given by vertical M5-brane instantons wrapping the elliptic fiber. While the possible 
superpotential contributions of such M5-instantons have been studied quite extensively starting 
with the seminal work of Witten [1] , many questions in particular related to charged zero modes 
are still quite poorly understood. Let us briefiy mention the different types of difficulties that 
appear in these investigations. 

A general problem arises from the fact that the world-volume theory of the M5-brane in- 
volves a self-dual 2-form field, whose dynamics do not admit a simple Lagrangian description. 
While a full covariant action has been known for some time |13] . this action includes an addi- 
tional auxiliary field and does not lend itself easily to a computation of the partition function 
which controls the low-energy superpotential contribution. Nevertheless, the partition function 
of the M5-instanton can be elegantly computed by considering first a simpler auxiliary action 
and then performing holomorphic factorisation [141115) . This procedure leads to several differ- 
ent candidate partition functions out of which the correct partition function must be picked. 
Performing this choice directly in M-theory is non-trivial. 

Our first task in this work is to compute the partition function of vertical M5-instantons 
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in a general F-theory compactification, based on the analysis in |14pi5|. The result can be 
compared, in situations with a Type IIB orientifold limit, with the E3-instanton analogue. We 
establish an explicit correspondence between the sum over gauge fluxes on the E3-instanton 
as described in [9] and the sum over 3-form fluxes in the M5 partition function. This match 
allows us to identify the correct partition function of the M5-instanton from the set of possible 
candidates. Our result is in agreement with the modular invariant partition function found for 
an M5-brane on a flat in |16fll8j and with the analysis of [19j for NS5-instantons dual to 
gravitational instantons in Type IIB. For the recent discussion of such gravitational instantons 
via M5-instantons see |2U] . 

Viewed microscopically, instanton zero modes arise in F/M-theory from wrapped M2-brane 
states, which can alternatively be described in terms of (p,q)-string junctions |2HI22). While 
their explicit study in a microscopic picture is difficult due to the lack of an explicit quantised 
membrane theory, some insights are provided by indirect arguments. First, the structure of 
neutral instanton zero modes can be determined using geometric techniques by studying the 
deformations of the instanton divisor [IJ, including situations with background fluxes (see e.g. 
[231428] ). In such a way, zero mode spectra can be evaluated in concrete examples with rather 
sophisticated methods |29H31j . The interplay with world- volume instanton flux has been studied 
recently in |12j in the language of E3-instantons in strongly coupled Type IIB theory. Other 
techniques to study instanton zero modes in F-theory include invoking duality with the heterotic 
string |22, 32J or anomaly inflow arguments (22]. Earlier work on instanton zero modes in the 
context of F-theory model bulling includes [53]; the structure of the so-called universal zero 
modes has been analysed in detail in [301IM]. 

The presence of charged zero modes can similarly be detected even without making use of a 
microscopic theory in terms of M2-branes. To gain some intuition it is beneficial to recall from 
the theory of E3-instantons in Type IIB orientifolds that the number of charged chiral zero 
modes is equivalent to a net U{1) charge of the instanton in the presence of gauge fluxes [3 l[ 5 |[ 6]. 
The same mechanism that is responsible for the gauging of the axions in the E3 worldvolume 
action is at work also in F/M-theory, as can be verified by a detailed analysis of the gauged 
supergravity |35l[36]. An important difference to the weakly coupled Type II theories was 
discussed in [36], though: In generic configurations some C^(l)s in Type IIB are massive even 
in absence of gauge fiux. These geometrically massive U{l)s are not present below the Kaluza- 
Klein scale in F-theory even though it is possible to formally make them visible by including 
non-harmonic forms in the dimensional reduction. In Type IIB the full set of U{1) selection 
rules - geometrically massive and massless ones - is necessary to account for all net charged 
zero modes; this suggests that in F-theory the obvious U{1) selection rules due to geometrically 
massless U{l)s are not sufficient by themselves either. In [36] it was in fact suggested, on the 
basis of the supergravity analysis, that in F-theory models with a smooth Type IIB limit the 
selection rules associated with massive U{l)s continue to be present. 

In Type IIB the chiral index that counts the net number of charged zero modes can be 
calculated by integrating the pullback of the 7-brane fiux onto the instanton divisor along 
the mutual intersection curve. In F/M-theory gauge flux is described by suitable G4-fiuxes. 
Euclidean M5-branes in the presence of G4-flux are subject to a Freed- Witten anomaly [37] 
whenever the pullback of G4 onto the divisor is non- vanishing. In [321138] it was argued that 
non-vanishing G4-fiux along the M5 divisor requires the inclusion of extra M2-brane states 
to cancel the Freed- Witten anomaly. These would be interpreted as charged zero modes, in 
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agreement with the Type II picture of OEIE] . A contribution of the M5-instanton not involving 
any charged operators is then possible only if the pull-back of G4 onto the instanton world 
volume vanishes; this serves as a criterion for the presence of charged zero modes and at the 
same time selection rule. 

An explicit analysis of this Freed- Witten anomaly hinges on recent progress in the explicit 
description of gauge flux via G4-fluxes in resolved Calabi-Yau fourfolds |39ft43] . The compu- 
tation of the pullback of G4 involves a search for 4-cycles on the instanton divisor which host 
a non-trivial G4-flux. We address this question in the context of globally U{1) restricted Tate 
models of the form investigated in |44j. The general picture emerging from our analysis is as 
follows: First, in the presence of massless U{1) gauge group factors, a special type of surface 
exists for generic complex structure moduli. The integral of the pullback of G4 over this surface 
within in the instanton computes the net U{1) charge of the instanton in agreement with the 
supergravity analysis. In addition to these U{1) selection rules, certain surfaces are present 
within in the instanton divisor which can be represented as algebraic surfaces only on special 
loci in complex structure moduli space; we will argue that nonetheless they lead to a non-trivial 
Freed- Witten constraint. These surfaces are related to the matter surfaces of the F-theory com- 
pactification, i.e. the fibrations of P^s in the elliptic fiber over curves in the base space where 
charged matter states are localised. The associated selection rules cannot be determined from 
massless C^(l)s in F-theory. 

This general analysis can again be compared to perturbative Type IIB results in situations 
where such a smooth limit exists |42] . The U{1) selection rules associated with the massless 
f7(l)s are found to match as expected. The selection rules due to the extra surfaces, on the 
other hand, encode what appears in Type IIB as the charge under the geometrically massive 
U{1) symmetries, confirming the expectations of |36J. 

The remainder of this paper is organised as follows. In section [2] we discuss the partition 
function of an M5-instanton employing the idea of holomorphic factorisation as in |14pi5]. In 
section [3] the result of this general computation is compared to the instanton partition function 
of Type IIB E3-brane instantons, which is reviewed in section 13.11 Amongst other things, 
the explicit match in section 13.21 identifies the correct holomorphic factor of the M5-instanton 
superpotential. In section [J] we analyse the interplay between M5-instantons and gauge G4-flux 
in F-theory. After a general outline of the problem in 14. H section 14.21 analyses the Freed- Witten 
constraint of vanishing G4-fiux pullback and discusses the possible types of surfaces that are 
relevant in detecting this anomaly. In section 14.31 we compare these results to the known 
selection rules in Type IIB orientifolds. Our conclusions and a summary of open questions are 
contained in section [5l Details of the M-theory effective action in the democratic formulation 
are relegated to the appendix. 



2 The partition function of vertical M5-brane instantons 

In this section we discuss the partition function of M5-brane instantons in M-theory compact- 
ified on a Calabi-Yau fourfold I4. Although the results of this section are valid for a general 
instanton, we will be interested mostly in situations where the fourfold I4 is elliptically fibered, 
vr : I4 — )• ^3, and where the M5-instanton wraps a vertical divisor 

Dm = n"^Dl^ (2-1) 
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with a divisor of the base space B3. In the F-theory limit the three-dimensional effective 
action of M-theory on an elliptically fibered I4 is uplifted to an effective action in four dimen- 
sions. In this limit an M5-instanton on a vertical divisor corresponds to a Euclidean D3-brane 
(dubbed E3-) instanton on the base divisor [1], more precisely to an E3-instanton wrapping 
the corresponding divisor on the double cover Calabi-Yau threefold X3. We will furthermore 
focus on the generation of a non-perturbative superpotential due to the Euclidean M5-brane in 
the effective action obtained in the F-theory limit. Since we are only analysing the structure 
of the partition function, we assume that the uncharged zero mode structure indeed allows for 
the generation of a superpotential. Finally, in this section we concentrate entirely on isolated 
M5-instantons, i.e. we ignore possible intersections of the M5-brane and the discriminant locus 
of the elliptic fibration|^ In the presence of such intersections non-trivial constraints due to the 
appearance of charged zero modes between the instanton and the 7-branes arise, which are the 
subject of sectional 

The worldvolume of an M5-instanton wrapped on a divisor Dm carries a chiral 2-form 
potential B with field strength Ti. In Euclidean signature, the Hodge star operator squares to 
— 1, *^ = —1, leading to eigenvalues ±i. Chirality of B refers to the fact that the field strength 
Ti is imaginary self-dual, *T-L = iH. Any 3- form Q on the M5-instanton can be split into a 
chiral and anti-chiral part according to 

Q± = ^(l^i*)Q, Q = Q+ + Q-. (2.2) 

Due to the chiral nature of B the definition of a covariant action involves auxiliary fields, 
which are necessary to impose self-duality of H [13]. One can bypass the associated technical 
complications by considering instead an action of the form 

5b = -2tt [ [{n + at*C3) A *(^ + aL*C3) + 6^ A l*C3] , (2.3) 

where Ti is not assumed to be self-dual; rather this condition has to be implemented in a 
suitable manner in all physical expressions as we will review below [13] • We have chosen the 
Hodge star operator such that J^_^^ % A *% is negative definite, which determines the overall 
sign of the action. 

Before we proceed let us briefly note that the Freed- Witten anomaly [37] actually rules out 
cohomologically non-trivial 4- form flux i*G4 7^ on free M5-instantons (with G4 = dC^). This 
can also be seen by simply considering the equations of motion of B from ()2.3p . The anomaly 
due to non-trivial flux must be cancelled by the inclusion of suitable M2-branes ending on the 
instanton cancelling the tadpole for B. As stressed above, in this section we only consider 
isolated M5-instantons with z.*G4 = 0. We will discuss the inclusion of background flux on the 
instanton in section 

The idea of [14j to obtain the chiral partition function from the non-chiral action (|2.3p is 
to compute the partition function for the full action ()2.3p in a fixed Cs-form background. This 
partition function is given by the path integral 

Zb = j VB e-^B. (2.4) 

^More precisely we assume there are no intersections other than with the Ji-locus of the discriminant as is 
necessary for the instanton to generate a superpotential, i.e. to form an 0(1) instanton in Type IIB language. 
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To avoid redundancies in the integration one has to omit S-field configurations which are pure 
gauge including those which correspond to large gauge transformations. In the absence of G4 
fluxes, the large gauge transformations correspond to a shift of B by an element of H'^{Dm,^)- 
The case L*Gi ^ 0, where this is broken to a discrete subgroup, will not be discussed until 
section 14.11 In the situation of trivial fluxes as considered here the path integral thus in 
essence splits into an integration over the non-closed 2-forms and a sum over non-trivial fluxes 
no G H^{Dm, Z)- The quantity (H^D takes the form of a sum of terms which are each a product 
of a factor containing only the self-dual part of the 3-form flux "H"*" and its conjugate defined in 
terms of T-L^ . In [14J, the partition function for the theory of a chiral 2-form B is identified with 
the chiral part of one particular of these summands. As we will see, it is possible to directly 
determine this summand by comparison with the Type IIB result for the superpotential of a 
D3-brane instanton. 

One comment is in order: The above analysis only accounts for the bosonic part of the 
M5-action involving the ;S-fieldll To fully account for all degrees of freedom one must specify in 
addition the superembedding of the M5-brane worldvolume and also the fermionic completion 
involving the superpartner of the S-field. These parts of the M5-action have been studied e.g. 
in [23H28] . In addition to the expression ()2.4p the full path integral then involves the integral 
over the bosonic modes from the embedding and the fermionic zero modes. Both are well-known 
to be crucial to determine if a superpotential is generated. In the sequel we concentrate only 
on the piece (j2.4p of the partition function, assuming implicitly that the pre-requisites for a 
superpotential contribution, as first specified in [T], are met. 

In order to identify which of the two factors contains the contribution of H.^ or "H^ it is 
helpful to choose the coefficients in (|2.3|) in such a way that only the anti-self-dual part C~ of C3 
couples to n. This ensures that T-l~ is decoupled from C3, so after performing the factorisation 
described above the desired chiral partition function encoding the contributing of T-L^ will be 
the factor involving C3. Expanding C3 = + in ()2.3p it is easy to see that 7i couples only 
to C~ provided b = —2ai. The numerical value of the constants a and b can be determined from 
the condition that the partition function matches the expression obtained for a D3-instanton 
in the Type IIB limit of F-theory. As we will see in section [3^ this leads to 

b = -i, i.e. a = 1/2. (2.5) 

Imposing these conditions and suppressing the explicit pullbacks the part of the action that 
encodes the dynamics of B (modulo self-duality) and the coupling to C3 is therefore given by 

Sb = -2tt [ [HA*n-2inAC- + ]c3A*C3]. (2.6) 

In addition one must include also the couplings to the remaining fields of the M-theory back- 
ground, in particular to the 6- form potential Cg dual to C3. The dimensional reduction of 
democratic formulation of M-theory involving both C3 and Cg is worked out in appendix [XI 
The complete action we consider is 

Sm5 = 27r(Vol^,, +i j c^) + Sb. (2.7) 

^This corresponds, in the language of D3-brane instantons in Type IIB, to the degrees of freedom descending 
from the instanton gauge field, i.e. the bosonic piece of the Wilson line multiplet. 
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However let us stress again at this point that this is still only an auxiliary action as it does 
not take into account the self-duality of B. The true physical content of the theory is obtained 
from the partition function ()2.4p after holomorphic factorisation. 

One last remark is in order before we proceed with the computation of the partition function. 
The gauge invariant field strength appearing in (|2.3p is not dB, but the combination 

A = dB + ^i*C3. (2.8) 
In general, gauge invariance under a gauge transformation of C3 requires an associated shift of 

B m, 

6Cs = -dx, SB = ^i*x. (2.9) 

One important special case occurs when the pullback t*x to the M5-instanton worldvolume 
Wm5 is non-vanishing but closed. This happens in particular when the transformation of C3 
is due to a gauge transformation of a three-dimensional U{1) gauge symmetry arising from an 
expansion of C3 = A"^ Auja into harmonic 2-forms loa on Y4. In this case <5i*C3 = because the 
gauge shift is purely along the extended three dimensions, where the instanton is pointlike. The 
corresponding shift of B in (|2.9p can in this case be undone by a large gauge transformation of 
B. However, we will stick to the generic case in the following and return to this special case in 
section SI 

While the combination A is thus invariant, the action ()2.3p itself is not invariant under a 
general gauge transformation of C3. Instead under the gauge transformation above one obtains 
via integration by parts 

5Si3 = -iTT [ X A G4, G4 = dC3. (2.10) 

Jdm 

This leads to the crucial conclusion [TJj that the associated partition function must transform 
as a section of a non-trivial line bundle over the space of C3-form configurations. Nonetheless, 
the full action Sm5 in (|2.7p of the M5-instanton is invariant under a gauge transformation as 
considered above. This is due to the fact that the dual field Cg transforms as 

5C6 = ^xAG4 (2.11) 

such that the field strength G7 = dCg + ^C3 A G4 remains invariant, see also appendix lAl for 
details. The transformation of Cg precisely cancels the variation ()2.10p such that the total 
action Sm5 is invariant. Although we have only demonstrated it for the pseudo-action ()2.7p . 
this gauge invariance under gauge transformations 6C3 = —dx is shared by the full covariant 
action [13]. The relevance of this gauge shift for M5-instantons in the presence of fluxes was 
also stressed in the recent PCT1I55]. 

We now turn to the explicit computation of the gauge invariant partition function 

Wm5 = jvBe-^^\ (2.12) 

The partition function splits into a sum over the saddle points of the action corresponding to 
a sum over harmonic flux configurations 1-Lq and a path integral over fluctuations of the i3-field 
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around these saddle point^. This corresponds to the decomposition 

n = nQ + d5B (2.13) 
in the action (|2.6p . The partition function becomes 

Wm5 = ^e-^«5[^ol / V5B e--5M5['^5e,«o]_ (2.14) 

Ho 

The prime indicates that in the second factor 1-Lq only appears in the kinetic cross-term 
Jp^^ %{) A *d{6B). In the semi-classical approximation the action in the exponent is expanded 
into fluctuations around the given background configuration up to second order. In particular 
the Hodge star operator splits into a piece *o determined by the background metric and a 
piece i that contains the metric fluctuations. This induces a dependence on the fluctuations 
associated with the embedding of the M5-brane and the bulk geometry. 

At zeroth order in fluctuations of the induced metric on the instanton the path integral over 
fluctuations of the gauge potential 6B decouples from the classical sum over fluxes, as the fluxes 
are harmonic and self-dual with respect to the background metric, 

fi*o^o = 0. (2.15) 

Therefore the kinetic cross-term in S']^jr,[d6B, Hq] vanishes and the Gaussian path integral J V5B 
reduces to an overall prefactor of the classical partition function 

W^lg = ^e-^"5[«o]_ (2.16) 
Ho 

At the first non-trivial order in the metric fluctuations this decoupling between Hq and 6B 
breaks down because ()2.15p no longer holds when *o is replaced by the fluctuation-dependent 
Hodge star *. Each classical term in the sum over T-Lq is then weighted by a flux dependent 
term J D6B e^'^^^sl'^^^'^o] These quantum corrections encode the dependence on the Kahler 
and complex structure moduli of the ambient Calabi-Yau as well as the deformation moduli 
of the instanton divisor. The path integral takes the form of a shifted Gaussian due to the 
coupling d5B A *{'Ho + i * C~) appearing in 5*^5 , so it is expected to lead to corrections that 
scale quadratically in the instanton fluxes. The quantum corrections can naturally be seen as 
the F-theory analogue of the one-loop corrections to the instanton gauge kinetic function in 
the Type IIB setting, which we will briefly discuss in section 13.11 As in Type II, extra zeroes 
of the superpotential can in principle arise from this Pfaffian over certain loci in moduli space. 
In Type IIB these are interpreted as due to vectorlike states of fermionic zero modes whenever 
the instanton hits another brane. Additional corrections to the partition function arise in the 
presence of non- vanishing G4-flux, which will be the topic of section WA\ For a recent discussion 
of G4-flux induced backreaction effects in the F/M-theory effective action of spacetime- filling 
7-branes see 



In the remainder of this section we focus on the computation of the classical piece (j2.16p . 
i.e. we will ignore the fluctuation-dependent weighting of e~'^^^5['Ho]_ Yot: simplicity of notation 



■^Note that by a gauge transformation of C3 we can assume without loss of generahty that u'Cs consists of a 
sum of harmonic and co-exact pieces. This imphes d* b'Cs = 0, so the equations of motion obtained from Sb 
indeed lead to harmonic Ho for vanishing G4-flux. 
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we drop the subscript in TIq and *o. In fact, reference [TS] computes the (classical) partition 
functioE0 

W{Q)= J2 e'^^-^^^*^exp{4m fnAQ) (2.17) 

for some Q G H^(Dm,^)- To understand the result and adapt it to the case of interest here, 
i.e. allowing for Q £ {{^(DmtC), we recall that for a 3-complex dimensional Kahler manifold 
Dm it is always possible to find a symplectic basis {Em,F^) of H^{Dm,'^), 

EmAF'' = 5Z, (2.18) 

Dm 

I Em/\En = 0= [ F^AF^, M,N = l,...,h^{DM), 
JDm JDm ^ 

such that any form Q G H^{Dm,'^) with integer periods can be expanded into {Em-,F^) with 
integer expansion coefficients. The Hodge star operator * allows us to express, say, F^^ in terms 
of E'at and ^Ejsj, 

pN ^ ^Mw^^ ^ y^'^^(*i^^). (2.19) 

Using (j2.18p it is easy to see that the matrices X and Y are symmetric, and that for our choice 
of Hodge star operator Y is negative definite. In terms of the complex matrix 

^MN ^ ^MN ^ -yMN (2.20) 

one can define a self-dual/ anti-self-dual complex basis of H^{Dm) as 

E+=- hm Zmn {F^ - Z^'^ Ep) , E~, = '-Im Zmn {F'' - Ep), = ±iS± , 

where IuiZmn = (ImZ*^^)~^ . An element Q G H^{Dm,C) can then be expanded into (anti-) 
self-dual components as 

q = q^ + q_ = qM e+ + gM^- _ (2.21) 

For a real form Q the two terms are related by complex conjugation such that {Q^)* = Qj^- 
Slightly adapting the result of [15] for the path integral ()2.17p . one find^ 

M^(Q):^ J]G[;](-Z,-Q+,-Q_) G[a(Z,Q_,Q+). (2.23) 



*We expect that in some cases there is an F-term condition which actually restricts the partition sum to run 
over only a sub-lattice of H^(Dm,Z), analogous to the well-known F-term condition for the flux on E3-instantons 
in Type IIB. This would not alter the formal presentation of the results below apart from replacing H''{Dm,Z) 
by the appropriate subspace. Therefore we ignore this subtlety for now and return to it at the end of section [3721 
when we have established the correspondence with Type IIB. 

^This holds up to an anomalous prefactor that will cancel in the full quantum computation [TS] . Note 
furthermore that in [15] Q is a real form; thus W{Q) is real and the factors appearing in 1)2. 23p are related by 
complex conjugation 



W{Q) ~ Yl ©SK-^. -<?-) . Q e H^Dm, R). (2.22) 



However we will need to be more general. 
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The theta-functions 

ee](Z,Q+,Q_) = expgQflmZMiv(Q^-Q^)] (2.24) 
X exp + a)MZ^'^ik + a)N + 2{k + a)M(Qf + /3*^))] 

involve a particular choice of ^/i'^(DA4-)-dimensional vectors aM,P^ with entries 0, ^ that label 
a choice of a line bundle on the intermediate Jacobian H^{Dm, C)/ H^{Dm ,'^) of the M5-brane. 
Such line bundles admit a holomorphic section unique up to rescaling, which is the corresponding 
theta-function [Hj. If the M5-brane world- volume is a spin manifold, the theta-functions are 
in one-to-one correspondence with the inequivalent spin-structures on Dm- 

The general prescription to construct the correct line bundle, and hence to extract the 
correct summand in ()2.23p . was given in [H]. However, this procedure is quite difficult to 
perform explicitly. For M5-branes which correspond to lifts of E3-instantons from Type IIB 
theory, we will see that one can identify the correct theta-function directly by comparison with 
the partition function on the IIB side, without going through with the construction of the line 
bundle. 

Finally let us note that the theta-functions appearing in (I2.23P are respectively holomorphic 
and antiholomorphic when viewed as sections of a line bundle on the intermediate Jacobian 
endowed with the covariant derivatives 

^ = ^ - iQ'^lraZ^M , ^ = J^- i^^Im^^M , (2.25) 

in the sense that 

^e[;](Z, Q_, Q+) = = ^Q[;](-Z, -Q+, -Q_). (2.26) 

As couples only to Q- its contribution to the partition function is contained in the anti- 
holomorphic factor which is annihilated by jjqM ; while the other factor contains the contribu- 
tion from the anti-self-dual part T-L-. 

We are now in a position to apply this result to the action of the M5-instanton 

Sm5 = 27r(VolM5 + iJcQ) + SB, (2.27) 

with Ss given by ()2.6p . The total classical partition function, containing also the undesired 
contribution from the anti-self-dual part of the field strength Ti-, is then given by 



•He h3{d 



where W{Q) is defined in ()2.17p . The classical partition function of the chiral 2- form is then 
obtained by picking one of the anti-holomorphic factors from the sum in ()2.23p and is given by 

= e-2^(VoiM5+i/C6)^g] (2.29) 
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for some choice of a and /3, with 

fcMG Z 

(2.30) 

For future convenience we have reinstated the parameter h. In section [3?2] we wih be able to fix 
the value both of h and of a m and by comparison with the partition function of the Type 
IIB E3-brane instanton, allowing us to identify the physical partition function of the chiral 
scalar from the set of candidates. 



3 Fluxed 0(1) D3-instantons in Type IIB orientifolds 

In this section we compare the structure of the M5-partition function with the non-perturbative 
superpotential induced by Euclidean D3-brane instantons in Type IIB orientifolds. We begin 
in 13.11 with a recollection of the moduli dependence of the fluxed instanton superpotential as 
described in [9j. This is followed, in section [321 by an explicit match of the partition functions. 

3.1 Moduli dependence of the instanton superpotential 

A Type IIB E3-brane instanton is a Euclidean D3-brane wrapped along a divisor De va. a, 
Calabi-Yau threefold and pointlike in R^'^. X3 will be the double cover of the base of 
the elliptically fibered fourfold I4 defining our F-theory model, = X^/a. The corresponding 
projection is p : X^ — )■ B3. In this paper we will focus on 0(1) instantons |46H49j . which wrap 
a single, irreducible divisor De that is mapped to itself as a whole without being pointwise 
invariant under the orientifold action, i.e. De = D'^ = a*DE^ 

As discussed in [HllIO], 0(1) instantons can carry worldvolume flux with negative parity 
under the orientifold action [Te = —o'*J-'e)- We denote the gauge invariant combination of the 
world- volume flux and the Kalb-Ramond i?-field by J^e = 2T:a' J-e — B. For such a conflguration 
the D-term constraint J A Te = is trivially satisfied and does not constrain the world- 
volume flux. However, an extra constraint arises from the Freed- Witten quantisation condition 
on the gauge flux 

J^E + i*B + ]^ci{KDE)^H\DE,'L). (3.1) 

For an 0(1) instanton the canonical class is even with respect to the orientifold projection. 
Thus the orientifold even part of the i?-field must cancel the half-integer contribution for a 
non-spin divisor as Te is odd under the orientifold action. 

The action of an 0(1) E3- instanton on the divisor De C X^ depends on the flux Pe as well 
as the chiral fields [SOtlST] 

= c^-Th", a = l,...,/i?:^(X3) (3.2) 

Ta = ^ICap-yv'^v'^ + i (^Ca - ICabcC^b''^ + ^TlCabcb^b'' , a = 1 , . .. , h]^^ {X3) , 

®If the instanton wraps a non-invariant divisor it is denoted a f/(l)-instanton, and one must consider the 
instanton along De and the image-instanton along D'e separately. The two cases differ qualitatively in the 
number of neutral zero modes, as in the case of the 0(1) instanton two of the universal fermionic zero modes are 
projected out by the orientifold action (see [7| and references therein for details on the zero mode structure). 
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where r = Cq + ie~'f' represents the axio-dilaton. The moduH c°, 6" and arise from the 
expansion of the fields C2, B and J into the bases {wq} and {coa} of the orientifold even 
and odd cohomology groups H^^ {Xs,1^) and H^^ (X^jX), respectively. Cq in turn is given by 
C4 = CqO;" + where {a;"} is the basis of i?i'^(X3,Z) normalised by 



/ u^Aoj^ = 5i. (3.3) 



Similarly one has a basis of i?^'^(X3,Z) which is dual to the {coa}- Finally we have also 
used the triple intersection numbers, written as usual in the supergravity limit as 

ICabc = / AWB Aa;c G 2Z, A,B,C = a,a. (3.4) 

Note that these triple intersection numbers are even integers due to the fact that we have chosen 
bases with definite orientifold parity. 

In order to write down the instanton action it is helpful to decompose the instanton flux as 

27ra'-FB = ^|t*wa + ^^, (3.5) 

where denotes the variable flux lying in the orthogonal complement of l*H^{X^) in -f/^^(-DEi3- 
For a U{1) instanton this expansion would include also positive parity fluxes. 
When computing the action of a D-brane or instanton one must take care of a factor of 1/2 
difference between the descriptions on the base -B3 and the double cover X^. For a general 
brane and image brane pair with Da 7^ one defines the combinations = Da U (±1)^), 
where the minus sign denotes a reversal of orientation, and introduces the wrapping numbers 

C%= [ u^, C%= [ (3.6) 

Jd+ Jo- 
in terms of the physical theory defined on the brane and image brane are descriptions of 
the same object, so the physical action is obtained by adding the actions of the brane stacks on 
Da and D'j^ and dividing by 2. In the case of an invariant object De = D'^ such as our 0(1) 
instanton one defines -D^ = De, which implies 

[De] = C^u;^ in X3 (3.7) 

for the 0(1) instanton. Nevertheless the physical action is still obtained by computing the usual 
brane action on De in X^, and then dividing the result by 2. This takes into account the fact 
that on the double cover all intersection numbers are twice those computed on the baseH 

For a given configuration of instanton flux, the physical action of the 0(l)-instanton is [9] 

Se = n{c%{T^ + i^Ea)+i^l), (3.8) 



^Variable flux is flux lying in the kernel of the Gysin map defined on H^{De)- Viewed more geometricaUy, 
the existence of such flux requires the existence of non-trivial 2-cycles on the instanton divisor that are trivial as 
2-cycles in the ambient space X3. 

*While it may seem unnatural to include it for a single invariant brane on De, this factor of 1/2 can also be 
seen to be necessary e.g. for consistency upon recombination of a (7(1) instanton-image-instanton pair into an 
0(1) instanton. 
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with 

A_Ba = K-abc + -^l^abc^E ^E^ ^'e = / -^E ^ ^E- (3-9) 

^ JDe 

Note that the universal contribution of the 6"-moduh is already encapsulated in and G"^ 
given in (|3.2|) so that only the actual gauge flux Fe (as opposed to Fe ) appears in lS.Ea- 

If the instanton divisor De intersects any divisor Da wrapped by one of the spacetime- filling 
7-branes, the instanton flux can lead to a chiral spectrum of charged zero modes between the 
instanton and the 7-brane stack or, equivalently, to a non-trivial instanton charge under the 
diagonal U{1)a realised on the brane stack. For the time being we ignore this effect in the 
same manner as we ignored the interplay between the 7-branes and the M5-instanton in the 
F/M-theory formulation. We will turn to this question in section HI 

As in M-theory the full partition function involves a sum over admissible fluxes Fe - the 
classical partition function - together with a path integral over the fluctuations 6 A of the instan- 
ton gauge field configuration around these saddle points of the action. Here we will compute 
only the classical part of the partition function which will be matched with the corresponding 
results for the M5-instanton obtained in the previous section. However, let us first briefly com- 
ment on the structure of the quantum contributions to the partition function from the Type 
IIB perspective. 

The discussion around (j2.16p applies in an analogous manner after replacing the 2-form B 
by the instanton gauge potential and the 3- form strength Ti by the instanton fiux Fe- In this 
supergravity picture not only the dependence on the Calabi-Yau moduli, but also on the brane 
moduli associated with the D3- and D7-branes present in the compactification is encoded via 
their backreaction on the metric. Specifically, the corrections to the holomorphic piece of the 
instanton partition function (which is relevant for the superpotential) induced in this manner 
by D3-branes and by fiuxed D7-branes have been discussed in [52] and [53] (see [MIES] for 
more general aspects of this moduli dependence of the instanton superpotential in terms of 
the supergravity backreaction). These articles consider E3-instantons with vanishing instanton 
flux. From the IIB version of the discussion around (j2.16p (i.e. after replacing H by Fe) it 
is clear that the corrections will in general depend quadratically on the instanton flux. This 
is also in qualitative agreement with [53] , which argues for a correction to unfluxed instantons 
that depends on the induced D3-charge of fluxed D7-branes wrapping a distant brane divisor 
S. This induced D3-charge is of course quadratic in the gauge fluxes on the D7-brane. By 
symmetry this in turn suggests that corrections in the presence of suitable D7-branes should 
also have quadratic dependance on the instanton flux. 

Finally recall that the quantum corrections to the instanton partition function can also 
be determined by a suitable stringy calculation of correlation functions in an instanton back- 
ground. The backreaction of the instanton and other D-branes on the background geometry 
corresponds directly to corrections induced by tree-level diagrams for closed strings interacting 
with the instanton. These admit an alternative description as one-loop open string diagrams of 
either Moebius strip topology with only a boundary on the instanton or annulus diagrams with 
boundaries on the instanton and other D-branes. Indeed such open string one-loop amplitudes 
are known to describe the instanton 1-loop Pfaffian [3l[56]- The relevant annulus diagrams that 
contribute to the holomorphic instanton superpotential have been identified in |57H59j . These 
are equivalent to the corrections to the holomorphic gauge kinetic function [60]. Again these 
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results suggest a correction that depends quadratically on the instanton flux. 
With this in mind we now turn to the classical partition function function 

TFl'g = ^e-^^[-^^l, (3.10) 

which is the direct analogue of the M5-instanton expression (|2.16p . It is helpful to rewrite this 
classical partition function in a slightly different manner such that in particular the integer 
quantisation of the fluxes required by the Freed- Witten condition (|3.ip is manifest. To this 
end we must take into account a minor subtlety related to the expansion of the pullback fluxes 
in ()3.5p . Note that the forms uia were chosen as an integral basis of //^'"'^(Xa, Z) such that an 
integral form on X3 could be expanded into this basis with integer coefficients. However, for 
general the corresponding statement does not hold for the pullbacks L*u]a as a generating 
system for l* H^}''^\X:i) n H^_^'^\De,Z), i.e. the J"^ in p.Sp are not necessarily integers even 
though the flux itself is an integer form. 

To facilitate comparison with the M5-instanton partition function we would like to choose 
a basis {ujm} of l* H^'^\X3) n H^j^'^\De,'^) in which the expansion coefficients are integer. 
At this point we must again take into account a subtle factor of 2 arising from the process of 
orientifolding. The Freed- Witten anomaly for the physical theory of unoriented open strings 
on the orientifold quotient requires the integral of the flux over any 2-cycle on the physical 
instanton divisor on the orientifold quotient of to be integer. This implies that over the 
uplift of this curve on the double cover the integral of I'kcJ Te is an even integer. Hence if 
we want 2txoI Te = J~'^^^m with integer expansion coefficients J-™" £ Z we must require on the 
double cover X^ 



/ 

JDp 



The new basis forms Um can be related to the pullbacks (which we suppress in the following) 
of the uja by some matrix 

= M>„. (3.12) 

The matrix M characterises the embedding of the divisor [De] = C^uoa wrapped by the 0(1)- 
instanton, and satisfies the useful relation 

C%lCo.ab= I u;aAL^b = 2M™M,"5„„. (3.13) 
JDe 

To define a full basis of H^^{De,1j) we extend this by forms iOm spanning the orthogonal 
complement of l*H^'^\x^). This allows for an expansion of the variable fiuxes in (j3.5p as well. 
The forms are by definition orthogonal to the Um and are also taken to fulfill 

/ A Wft = 25rhh- (3-14) 

JDe 

An integer quantised flux 2Tia'J-E can then be expanded into this basis as 

27ra'^E = ^'^iVCo;^ + {rToJrh = J'^'^M , G Z . (3.15) 
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Here and in the following the index M runs over the fuh basis M = m, rh. Similarly, one has 



(3.16) 



With the help of these relations we may rewrite the classical partition function of the 0(1) 
instanton p.lOp as 



Ed, 



exp 



X exp 



ITT 



T — T 



3.2 Matching the M5 and E3 partition functions 

To match the partition functions of the M5- and E3-instantons we need to match the sum 
over 3-form fluxes H G H^{Dm) in M-theory to the sum over admissible 0(1) instanton fluxes 
Fe G H^^{De) on the IIB side. An important ingredient is the relation between the two 
cohomology groups in which the fluxes take their value. Recall that the M5-instanton wraps 
a divisor Dm of the F-theory fourfold I4 : ^ I4 — )• S3 which is itself an elliptic fibration 
Dm '■ T"^ ^ Dm The Type IIB orientifold is defined on the Calabi-Yau threefold 

X3 obtained as the double cover p : — )• i3 of the F-theory base B. The divisor De C 
wrapped by the Type IIB instanton is the preimage De = p*{D\j) of D\j. 

For simplicity we only consider Type IIB instantons which do not admit non-trivial Wilson 
lines, such that h^{DE) = h^{DE) = 0. In this case generic elements of H^{Dm) arise from 2- 
forms on De with negative orientifold parity. Given a 2-form lo G H^{De) one can schematically 
see the corresponding 3-forms as arising from the wedge product with one-forms dx, dy G 
H^{T2) on the elliptic fiber [21] (see also i32j), 

Li; — )• w A dx, Cl! — )• w A dy. (3.18) 

The negative monodromy of dx and dy around the orientifold plane cancels the negative parity 
of u) such that the product gives rise to a well-defined 3-form on D^j. This qualitative picture 
was confirmed in an an explicit example in [21j . where it was found that H^'^^^\Dm) is the 
uplift of h'l''^\De), while HC^'^^Dm) receives contributions from the uplift of H^'^\De) 
and H^'^\De)- We will denote the two subspaces of H^'^''^\Dm) related to H^'^\De) and 

to H^^'^\De) by H^^P{Dm) and hH^XDm), respectively. Finally, H^'^'^\Dm) receives a 
further contribution from negative parity one-forms on the intersection of the instanton with 
D7-branes, counted by h}_{DE H Dl). We will always assume that this possible contribution is 
absent, h^_{DE n 1)7) = [21]. 

There is an important caveat to the above relation between H^^\Dm) and H^'^\De) that 
follows from the analysis in [35] '■ The argument (|3.18p for the uplift of the latter fails if uj has 
non-vanishing pullback onto an intersection of the instanton with a D7-brane on a divisor D^ 
above which the elliptic fiber degenerates. As DeDDa is non-trivial as a 2-cycle in the ambient 
space X3 this is possible for forms in l* H^'^\x-i) n H^'^\De) which arise as pullbacks from 
the ambient space. In the language of section [3T] the (l,l)-forms for which the uplift ()3.18p can 
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fail to lead to an element of H^{Dm) are therefore the forms L*uJa with 

/ / i*iOa = ljCabaC\C%, where [De] = C^UJa, [Da] = liC^LO^ + C%iOa). (3.19) 

JDeHDa ^ ^ 

Note that this is not in contradiction with the results of |21j , as in the example considered there 

By the discussion above we see that the uplift of H^}'^\De) can become problematic only 
if there are D7-branes with odd wrapping numbers C\ ^ present in the model. It was argued 
in [36] that the diagonal U{1)a of the D7-brane stack, which is massive by the geometric 
Stiickelberg mechanism, in this situation is described in M-theory by certain non-closed forms. 
To circumvent this subtlety we assume for now that = for all D7-brane stacks present, 
such that no geometric gauging occurs. In particular we therefore have an uplift 

H^}^^\DE)^Hg^\DM). (3.20) 
We will comment on the more general case at the end of section 14.31 

We are now in a position to explicitly match the superpotential contributions generated by 
the E3- and M5-instantons. Let us first assume that the E3-instanton is wrapped on a rigid 
divisor, i.e. H^_^''^\De) = 0. In light of the preceding discussion we then have an isomorphism 
between H^'^'^\Dm) and H^'^\De)- Note that the divisor D^^j^ in the base will intersect 
the discriminant locus A at least along the piece of /i-locus which is the uplift of the orientifold 
plane, as is necessary for the instanton to be of 0(l)-type. It can also intersect further com- 
ponents of the discriminant locus corresponding to stacks of D7-branes. However, as we have 
assumed that all branes are wrapped on orientifold-even divisors the negative parity 2-forms in 
H^'^\De) have no support on the 07-plane and the D7-branes. Hence we expect the uplift 
()3.20p to hold despite the fact that the elliptic fibration degenerates over A n D^^^ . 

As in section 13.11 we must take care regarding the quantisation condition of ^-flux along 
forms that arise by pullback of the ambient space basis Oa, G H^{Y4). This basis is the uplift 
of the basis uja of H^{B^) according to the prescription ()3.18p . As before these pullbacks do not 
necessarily form an integral basis of H^{Dm,'^) H l*H^(Y4). Hence they cannot immediately 
be identified with a subset of the 3-forms Em, introduced in ()2.18p as an integral basis of 
H^{Dm,'^)- One expects that the Em, arise in a similar manner to (j3.18p from the forms 
lom introduced in (|3.1ip resp. (j3.14p . Hence we can similarly split M = m, m, where E^, F"^ 
span the space of forms obtained by pullback while E^, F^ span the orthogonal complement. 
In full analogy to p.l2p one also gets 

i*aa = M^Em, i*5abP'' = M^SmnF''. (3.21) 

Note that this is consistent with the symplectic structure of (|2.18p and (|3.13p if one uses the 
intersection properties of the given in ()A.9p . 

Now comparing the M5-instanton partition function ()2.29p with the partition function of 
the E3-instanton given in (j3.17p one finds a perfect match with the help of the identification^ 
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Note that of course C± has expansion coefficients only along the forms Em, F"^ as it arises by pullback, or 
in other words C± = 0. We should also point out that the diagonal form of z'^'^^ is a consequence of the chosen 

pM 



basis of forms ojm giving rise to Em, F , which were taken to obey ()3.1ip and ()3.14|) 
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In order to obtain this match two non-trivial assertions must be made. Firstly, comparison 
with Type IIB fixes the parameter 6 = — i in the M5-brane action ()2.3p . as anticipated in the 
discussion before (12.51). 



Second, we have to set the parameter which appear in the ^-functions in (|2.29|) to the 
values um = = thereby fixing the correct expression for the M-theory partition function. 
A non-trivial value for the half-integer vector om would correspond to a shift in the Preed- 
Witten quantisation condition on the instanton flux J-'e- However, this does not occur for 0(1) 
instantons, as in this case ci{Kj:)^) G H^{De)- In addition there appears to be no room for a 
non- vanishing vector /3. As discussed the procedure to correctly identify aM,P^ purely within 
M-theory is non-trivial and quite involved in explicit setups [13] . Our identification a = = /3 is 
in agreement with previous analyses of an M5-brane wrapped on a flat torus. In this situation 
the result has been deduced by requiring modular invariance of the partition function |17j, 
by comparison with D4-brane results p^J or by a direct computation using a Hamiltonian 
formulation [16]. It is also in agreement with the study of NS5-instantons performed in |19j . 
In fact, it has been pointed out in [T7] that in the case where the worldvolume of the M5- 
brane can be written as X M4 with a simply-connected manifold M4, the requirement of 
modular invariance suffices to determine the partition function uniquely. However, our result is 
applicable to a much larger class of problems including degenerations of the torus fiber. Recall 
that the existence of such degeneration loci in the instanton worldvolume are necessary to be 
able to interpret the instanton as being of 0(1) type. In our case the geometry of the instanton is 
unconstrained apart from the requirement that the divisor Dm be a vertical divisoi0. Note that 
the existence of a Type IIB limit to compare with only implies certain topological constraints 
on the discriminant locus of the elliptic fibration [l2] . As far as the structure of the partition 
function ()2.29p is concerned, this is not a restriction. 

The relationship between the and G™" can also be obtained directly by examining the 
expansion of C3, 

abh. \ JT- _L A/T^f^o. _ =ra6r . n 771+ 



m 



= M:^G-E-+M:'G''E+, (3.23) 

confirming the identification C!!* = G™. Finally let us also check that the classical prefactor of 
the M5 action also matches precisely with the corresponding Type IIB expression. Expanding 
[Dm] = C'^Ua we get with the definitions of appendix \K\ 

- 27r _^^^(^^' + iC^) = -^C% (2V„ + ic^) = -ttC% {w^ + i(c, - ^/C,ac<5^''c"6b)^ , (3.24) 

which in the F-theory limit reduces precisely to — vr (^^C'j^lCai3'yV^v'^ + iC'^{ca — ^lCaabC°'b^ 
This reproduces the moduli dependence of the Type IIB E3-instanton action using the natural 
identification of Cq, c", 6" whith their IIB counterparts. Note that the shift between and 
determined in appendix [X] is crucial in obtaining this exact match. 

Let us now consider the effect of relaxing the rigidity condition on the instanton. Recall that 
in this case H^{Dm) splits into the subspaces H^^'^\Dm) © H^av\DM) and H^r\DM) (plus 



^"Relaxing the assumptions of rigidity and absence of Wilson lines would lead to more complicated notation 
as the classical partition sums would run only over suitable subspaces of H^{Dai), but would not change the 
qualitative results. See the discussion at the end of the present subsection for further details. 
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conjugates). These subspaces are the uphfts of H_' {De) and of H_' (De), respectively. 
The F-term super symmetry condition restricts the flux on the E3-instanton to take values in 
H^'^\De)- We expect that there exists a corresponding F-term condition on the M-theory side 

(2 1) 

that restricts the super symmetric flux on the M5-instanton to H^er (Dm)- In this case also /cm 
appearing in the M5-instanton partition function (|2.29p is a vector of dimension dim. H^l^ (Dm), 
and the identification in (j3.22p is unchanged. 

The F-term condition for the E3-instanton can be obtained from the superpotential term 

/ i^n^AJ^E, (3.25) 
JDe 

where denotes the contraction of a normal vector field to De with the holomorphic (3,0)- 
form 1^3. (|3.25|) allows for an obvious uplift of the form 

/ i^n^An, (3.26) 
Jdm 

which suffices to rule out unwanted contributions to H in H^^'^\Dm)- However it is less clear 
how contributions from -ff^or k^m) are ruled out. It would be interesting to investigate whether 
the mechanism is connected to the fact that in Calabi-Yau fourfolds the variations of do not 
span H'^^'^iYi) or whether further contributions to ()3.26p must be included, but this is beyond 
the scope of this work. 

In the above we have always assumed that h^{DEriDA) = 0. If one relaxes this assumption 
there must be additional contributions to (j3.26p which rule out this additional set of 3-form 
fluxes from appearing in the partition function, in order to have a match with Type IIB where 
the fluxes are in H^y^{DE). The same statement applies if we consider relaxing the assumption 
regarding absence of Wilson lines. However a detailed study of how these constraints arise is 
beyond the scope of this paper and would be an interesting subject for further study. 



4 Instanton selection rules in backgrounds with G'4-flux 

In this section we discuss the interplay between the M5-instanton and the 7-branes of the 
compactification due to non-trivial intersections of Dm with the discriminant locus. In Type 
II orientifolds the effect of brane-instanton intersections is well-known The open string 

sector gives rise to zero modes localised on the intersection curve which are charged under the 
brane gauge group. In the presence of non-vanishing gauge flux on the 7-branes, the spectrum 
of these modes can be chiral, resulting in a net charge of the instanton under the 7-brane gauge 
group. In [9j it was pointed out that also the instanton flux can lead to a chiral zero mode 
spectrum, and that the effect of gauge and instanton flux can cancel each other. These two 
effects have rather different analogues for M5-instantons: In F/M-theory 7-brane gauge flux 
is described by G4-flux, whereas we have seen that instanton flux is described by 3-form flux. 
The effect of G4-flux on the instanton partition function is the subject of section 14.11 As we 
will see and has already been discussed from various perspectives in |32y38|. absence of chiral 
zero modes due to gauge flux requires that the pullback of the G4-flux onto the M5-instanton 
vanishes, i*Gi = 0. In section HT2] we analyse this constraint in detail in the framework of |4ip42j . 
In section 23] we compare the flux induced instanton charges in the languages of F/M-theory 
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and Type IIB. This includes a discussion of massive U{l)s and also some speculative remarks 
about zero modes induced by instanton flux from the M5-brane perspective. 



4.1 G'4-flux and zero modes from M2-branes 

G4-fluxes form a crucial ingredient of phenomenologically interesting F-theory backgrounds as 
they are necessary to generate chiral matter spectra. As such the effects and explicit construc- 
tion of consistent G4 gauge flux as suitable elements of H'^{Y4) have received considerable recent 
attention in the literature, see e.g. [39tii3] for a selection of recent papers on the subject. 

To appreciate how the inclusion of G4 gauge flux modifies the M5-partition function we note 
that the action in ()2.3p implicitly depends on the pullback /,*G4. Integration by parts of the 
last term gives Ini J H /\ i*C-i = — 27ri J B A i*dC^. As always this should be read as a piece 
depending on the fluctuations of dC-^ and a piece involving the background flux G4 = (dCs). 
In the sequel we make this distinction explicit by writing 



Sb,G4 = SB-2m J BA L*G4 (4.1) 

with the understanding that S'g contains the fluctuation term upon integration by parts. It 
is this piece that we have considered so far. Note that the additional term is present only if 
i*G4 7^ in the cohomology of Dm, which we assume to be the case in the remainder of this 
subsection. 

Let us flrst point out one important consequence of i*G4 7^ 0: The coupling to G4 breaks 
the group of large gauge transformations of B (without shifting C3 or Cq) from {{"^{Dm,^) to a 
discrete subgroup, say, H'^{D]\j,'I,) (or an analogous one depending on the explicit quantisation 
of 6*^4). This implies that H"^ {D m -i^) / H"^ {D m ■,'^) must now be integrated over in the path 
integral over B. The naive superpotential for t*G4 7^ then derives from the integral 

j VBe~^'^'^^^"^°M+^S ^^)+^B,Gi\ = j VBe'^^'^'^i. (4.2) 

This, however, is incomplete for the following two reasons: First, as pointed out in [32], Sb,Ga 
contains a G4-dependent tadpole for B. As a consequence, the path integral J Pi3 e"^'^* 
is proportional to a delta function 5{i*Gi), which vanishes for non-trivial pullback of G4. 

We offer the following interpretation of this observation in the language of E3-brane instan- 
tons in the Type IIB limit: In the presence of non-trivial gauge flux on the intersection curves 
of the instanton divisor and the 7-branes extra zero modes Aa,Aft appear, which are localised 
on these intersection curves and are charged under the 7-brane gauge group |31[S1[^. Thus 
the partition function must include the integral over those zero modes Aa,Af,, and the direct 
analogue of the expression ()4.2p is 

I V\aV\^e-^^^^^l (4.3) 

In particular we can formally identify J T>XaT>Xi, with the new integral / VBe^""^ / ^^'•* 
taken only over H'^{Dm,^)/H'^{Dm,'^)- As we will see in section 22] this is further supported 
by the fact that the charge of this piece under massless U{l)s, if present, cancels the charge of 
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^-[2iT{\o\r)^j+i J C(i)+Sj3] ^ which now also exhibits a non-trivial shift due to the transformation of 
Cg. Without further modifications the quantity ()4.3p vanishes as the Grassmann integral is not 
saturated. 

Second, and independently of the previous argument, there is an intrinsic reason why the 
expression for the superpotential must be modified both in M-theory and in the Type II limit. 
For the M5-brane the equations of motion for B from (|2.3p . 



d*nocL*G4, (4.4) 

call for additional sources cancelling [6*^4] for consistency. In fact, as discussed in 08j, the M- 
theory version of the Freed- Witten anomaly ^37j directly requires the inclusion of such sources 
in the form of M2-branes ending on the instanton. These M2-branes cancel the flux-induced 
source term. The world- volumes of the M2-branes in question are given by 3-chains of the form 
Fj = / X 7j, where the 7^ are 2-cycles in Dm satisfying [H5] 

Y,l^ = P^D,Ai*G,] (4.5) 

i 

and / is a semi-infinite time interval such that dTi = {to} x in the M5-instanton worldvolume. 
PD Dj^j denotes the Poincare dual taken on the divisor Dm ■ 

The appearance of these M2-brane states can also be seen directly by considering the solution 
for ^ in a background with G4. This flux configuration will be sourced along PD^)^^ [t*G4]. In 
a supersymmetric configuration this field configuration is accompanied by a deformation of the 
M5-brane into the normal direction. This deformation can be identified with an M2-brane 
ending on the brane at PD^^j^ [i*G4] [ST]- In particular this picture suggests another equivalent 
way of thinking about the zero modes of the M5-instanton, namely as the zero modes describing 
the deformations of the solitonic ?^-field solution sourced by the G4-flux. 

The upshot of the discussion above is that when calculating any correlation functions in 
the M5-instanton background we must include the operators describing these M2-brane states, 
which are proportional to the exponential of the action of the M2-brane [38] . This leads to |62] 



VM2,i oc exp 



2^i ( ^ / C3 + / B 



(4.6) 



These states therefore contribute to any instanton-generated coupling in the presence of non- 
trivial G4-flux on the instanton. Such a coupling then takes the form 

Wm, JvB llVM2,i e-^^'K (4.7) 



We suggest an interpretation of these vertex operators by comparing again the situation for 
M5-branes with the Type IIB limit: The effective action of the E3-brane instanton contains 
interaction terms between the charged zero modes \a,\ and charged open string modes 
in the sector of open strings stretching between 7-brane stacks a and h. These operators make 
the saturation of the Grassmann integral possible. A superpotential term generated by such an 
instanton then schematically takes the form [3l[5l[6] 
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A non- vanishing result therefore hinges upon the insertion of the vertex operator 

in the path integral ()4.3p . The M2-brane states are therefore the M-theory analogue of the 
charged open string states appearing in ()4.8p 

^Af2,i = 6^^°*"*^^ (4.10) 

Note that in both pictures the full vertex operator is gauge invariant in particular with respect 
to extra C/(l) symmetries, if present, as discussed in the next section. 

4.2 Selection rules for the absence of chiral charged zero modes 

From the discussion in the previous subsection it is clear that a necessary condition for an M5- 
instanton to contribute to a superpotential coupling without involving any M2-brane vertex 
operators is 

i*G4 = 0. (4.11) 

While this well-known form of the Freed- Witten condition is conceptually simple, it can never- 
theless be non-trivial to evaluate in practice as it requires checking that the integral of G4 over 
every 4-cycle in Dm vanishes 

4.2.1 General picture 

To analyse the selection rule in detail let us first recall the explicit description of G4 gauge 
fluxes F/M-theory. We are working in the framework of an F/M-theory compactification on 
an elliptically fibered Calabi-Yau fourfold I4 whose singularity structure is responsible for the 
appearance of a gauge group G along the 7-branes. This may include also Abelian factors of 
U{1)a gauge groups. A proper definition of the low-energy effective theory is possible after 
resolving the singularities of Y4 and by working on the resolved fourfold Y4. The singularities 
associated with the non- Abelian piece G' of the total gauge group are resolved by introducing a 
set of rk(GO resolution divisors Ei. One way to realise extra U{l)s associated with non-Cartan 
generatorqij in Calabi-Yau fourfolds has been described in |44) : this leads in general to a set of 
further resolution divisors Sa- 

Gauge flux on I4 is encoded in a class G4 G H^'^''^\Y4) vertical to the pullback of any two 
base divisors and to the section of the elliptic fibration. The cohomology i?(^'^)(l4) splits into 

['22)'^ f2 2) ^ 

the primary vertical subspace ifver (^4) and the primary horizontal one -f^hor (^4)' defined 

as the complement of the first |63H66] . By definition, i^ver {Y4) is spanned by 4- forms that 

(2 2) 

factorise into two 2-forms. A special type of gauge fluxes in Hver (Yi) is of the form 

G4 = -T\ A WA, T\ G F(i'i)(i?3) (4.12) 

^^Note that even if (|4.1ip is satisfied, the inclusion of extra vertex operators, albeit with zero net charge under 
the 7-brane gauge groups, may be required, as is familiar from Type II instantons with vector-like pairs of charged 
zero modes. We will only analyze the above necessary condition in the sequel. 

^^These will be referred to as "non-Cartan U{l)s" in the sequel. 
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with an element of H^'^iXi) which is not in H^'^^B^) and not given by the fiber class. 
Such G4 is the flux associated with a U{1)a gauge symmetry whose potential arises from the 
expansion 

C3 = AaAwa + .... (4.13) 

This is true for Cartan ^7(l)s of a non-Abelian gauge group or for non-Cartan U{l)s alike. 
For Cartan fluxes, the 2- form w^ is the 2- form dual to a combination of resolution divisors Ei 
introduced in the blow-up of the non-Abelian singularity over a given divisor. For non-Cartan 
U{l)s the 2-form w^ is related to the resolution divisors Sa [S], where the exact form of w^ 
determines the precise normalisation of the U{1)a charges. Associated U{1)a fluxes have been 
constructed in 



Flux of the type G4 = —J-^a ^ represents only a special class of gauge fluxes. More 

(2 2) 

generic can also be the effectively vertical sum of products of 2-forms without 

factorising into a 2-form in H^''^{B^) and a 2-form in its complement in H^'^^iY^). In |40J such 
gauge fluxes were constructed on resolved fourfolds and related to the spectral cover approach. 
These fluxes are not associated with a massless U{1) in F-theory. As shown in in the 
Type IIB limit such flux can be identified with the linear combination of diagonal ^7(l)s which 
is massive by the geometric Stiickelberg mechanism, where in addition one must impose the 
D5-tadpole constraint!^ 

(2 2) 

Finally G4 can be an element of -ff^or (^)- particular, the Higgsing of a U{1) symmetry 
via brane recombination results in a geometry that does not exhibit the correponding 2-form wa 

any longer. As a result of this deformation 4- forms of the type D Awa become non-factorisable 

(2 2) 

elements in H^^^ . Examples of gauge fluxes described by these forms have been constructed 



in [39] (see also [H]). 

We now come back to an analysis of the constraint ()4.1ip . To evaluate it we consider 
integrals of the form 

/ L*G4 with ^(4) e Hi{DM)- (4.14) 

Since 1*0^ is obtained by pullback from the ambient Calabi-Yau I4, this integral can only give 
a non- vanishing result if C(4) is non-trivial in H/^iY^). By construction the fluxes are orthogonal 
to surfaces either lying entirely inside the base or wrapping the full elliptic fiber. 

As a result only two types of surfaces can lead to a non-zero integral ()4.14p . The first type 
of surfaces exists in the presence of massless U{1) factors (including the Cartan C/(l)s) and the 
associated selection rule is related to the net U{1) charge of the instanton. Since with each 
massless U{1)a symmetry comes a 2-form w^ G H^^'^iY^) as in (|4.13|) . one can construct the 
algebraic surface with dual 4-form 

[C^)] = -Dm Awa€ Hili^\Y,). (4.15) 

The surfaces [C'(4)] = —Dm A Ei are associated with the Cartan U{l)s within the non-Abelian 



In [36] it was conjectured that the uplift of such fluxes should be described by a set of non-harmonic fluxes 
subject to an analogue of the D5-tadpole constraint. This is consistent with the recent observations of [42j 
because the D5-tadpole constraint of [36] has the potential to effectively eliminate the non-harmonic parts. 
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group G' , but also the non-Cartan U{l)s give rise to corresponding surfaces. Indeed the integral 



L*Gi = - / Dm a wa a G4 (4.16) 



is in general non-vanishing. For fluxes which have been chosen such as to leave the non-Abelian 
gauge group G' unbroken, this integral vanishes by construction for the Cartan surfaces [C^4)]- 
Nevertheless a non-trivial constraint remains in the presence of extra non-Cartan U{l)s. Indeed 
the expression ()4.16p has a very intuitive interpretation from the point of view of the three- 
dimensional low-energy effective action obtained by compactification of M-theory because G4- 
flux induces a gauging of the shift symmetry of the fields Cq appearing in an expansion of Cg. 
As discussed in Appendix El the gauging of the fields Ca under a gauge transformation of a 
canonically normalised U{1)a is given by 

Aa^ Aa + cIAa 5A^Ca = -QaA^A- (4.17) 

The charge vector QaA is defined by 



= 2 / lOo^AwaAGa- (4.18) 
Jy4, 

The induced shift of the term J Gq in the instanton action is then simply 

V [cg = -^G^QaAA^ = - i Dm AwAAGi, (4.19) 



JY4, 



which is none other than ()4.16p . From the Type II perspective we are well familiar with the fact 
that a net transformation of the instanton action under a given U{1)a symmetry requires the 
introduction of charged operators O in the non-perturbative superpotential such that O 
is gauge invariant. As discussed in section ()4.ip such a mechanism is also required from a 
microscopic perspective. We see that the microscopic and the supergravity constraint match, 
as they must. 

The second type of surfaces are present also in the absence of a massless ^7(1). These 
surfaces lie within Dm even though they cannot be described as algebraic surfaces for generic 
choices of complex structure moduli. Similar phenomena have recently played important roles 
in the context of G4 gauge fluxes in [39H67j . Nonetheless they give a non-zero contribution 
in integrals of the form ()4.14p . One type of such surfaces is related to the matter surfaces of 
the 7-brane sector; another class can be related to the algebraic surfaces (|4.15p upon brane 
recombination, or Higgsing of the U{1)a, and can be described with the methods of [39j. In 
both cases the selection rule has no interpretation as the charge of a massless U{1) in M-theory. 
Note that the number of independent selection rules will in general be much smaller than the 
number of surfaces that can be constructed in this way. 

This second type of surfaces is more complicated because the dual 4-forms are no longer 
factorisable. For ease of presentation we therefore refrain from describing them in full generality 
but rather illustrate them in a concrete set of models, furnished by a global Tate model with 
gauge group SU{5) x U{l)x- The generalisation to more general groups (e.g. SU{N) instead 
of SU{5)) and to several U{1) factors will be clear. 
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4.2.2 Example: SU{5) x U{l)x 



Our treatment of the SU{5) x U{l)x Tate model is based on the analysis of refs. |4ip42j. whose 
notation we briefly summarise. Consider a singular Calabi-Yau fourfold ^4 given as a Tate 
model over a general base B^, 



The locus Pt defines a hypersurface in an ambient fivefold, which in turn is given by fibering 
the weighted projective space 1*2,3,1 with homogeneous coordinates x,y,z over B^. The Tate 
polynomials aj are sections of where K, is the anti-canonical bundle of i?3. They are chosen 
in such a way as to produce an SU{5) singularity in the fiber over the base divisor W : w = 0. 
This leads to [68l[69] 

ai generic, 12 = 02,1 if, as = 03,2 ■w^, a4 = a4,3U)^, 05 = 05,5 w^^- (4-21) 

Calabi-Yau fourfolds for F-theory compactifications of this type have been analysed intensively 
in the context of F-theory model building in the recent literature |70ft75] . Furthermore we 
take qq subject to the extra constraint a6,5 = such as to produce an additional U{l)x gauge 
group [H] . After resolution of the singularities associated with the gauge groups SU (5) x U {l)x, 
the resolved Calabi-Yau fourfold I4 is given by the proper transform of the Tate polynomial 
in a toric ambient fivefold X5. In addition to the usual coordinates x,y,z of the smooth 
Tate model and the coordinates of the base -B3, the homogeneous coordinates of this fivefold 
include the coordinates e^, i = 1, ... ,4 (as blow-up coordinates of the SU{5) singularity) and 
s as a consequence of the Abelian gauge group, together with a number of scaling relations. 
Associated with these resolution coordinates are the respective divisor classes Ei, ...,E4^,S. The 
first explicit resolution of these singularities in Calabi-Yau fourfolds has been provided in the 
toric examples of [721 173] (see also [76]), followed by an in-depth analysis of the resolution 
in particular of higher codimensions singularities in |40 p 4 H H3 l l77]. The resolved Calabi-Yau 
fourfold I4 is then described by the locus 



within X5, where eo is the coordinate associated with the proper transform of the divisor W. 
Finally the extra U{l)x factor is due to the existence of the 2-form 



which is related to the resolution divisor S arising in the C/(l)-restricted Tate model. The 
specific linear combination wx is chosen such that the U{l)x generator is orthogonal to the 
Cartan generators of SU{5) [41]. 

(2 2) 

There exist exactly two types of generic (i.e. base-independent) G4-fluxes in i^ver (^4) 
which do not break the SU{5) gauge group. In terms of the above divisor classes they are given 



Pt '■ {y^ + ciixyz + a^yz^ = + 02x^2;^ + 04x2;^ + a^z^}. 



(4.20) 



Pt ■■ {y^ 86364^ + aixyzs + 03,2 U Cq ei 64 

= ei 62 63 + 02,1 x^ z'^ s eo ei 62 + 04^3 x Cq e\ 62 64} 



(4.22) 




(4.23) 



by \MM 





(4.24) 




)i^i a£). 



(4.25) 
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It is understood that the parameter A and G H^'^{B^) are chosen in such a way that the 
fluxes obey the Freed- Witten quantisation condition 



(4.26) 



This constraint has been analysed in detail in [671176] . The fluxes (j4.24p and (j4.25p are what 
we will focus on in the sequel. 

Following the general discussion of the previous subsection, associated with the U {l)x gauge 
factor is the algebraic surface with dual 4-form 



According to the discussion around ()4.19p these expressions are the U{l)x charges of the in- 
stanton in presence of the corresponding fluxes, where the explicit factor of 5 has been included 
to account for the normalisation of the U{l)x boson associated with wx as defined in (|4.23p . 

The second set of surfaces over which we can integrate i*G4 is related to the matter surfaces. 
Matter surfaces consist of certain combinations of P^s fibered over curves in the base where the 
singularity type is enhanced. They have recently been analysed in detail in the context of elliptic 
fourfolds in [lOlllIllMllIZ! . We follow the approach and notation of [H]. In the SU{5) x U{l)x 
setup under consideration, there are three types of such matter surfaces, corresponding to the 
loci of singularity enhancement to SO{10) and SU{6) on the GUT brane as well as the curve of 
self- intersection of the /i-locus with an enhancement to SU{2). We exemplify the constraints 
arising from these surfaces by analysing the surfaces Cfg fibered over the curve of 50(10) 
enhancement Cio = H {a\ = 0} C -B3. Each resolution divisor Ei is fibered over W^. 
Some of these P^s in the fiber split into two or several P^s over Cio- The P^s in the fiber over Cio 
group into certain linear combinations of curves which can be identified with the weights of the 
10-representation of 50(10). By this one means that an M2-brane wrapping one of these linear 
combinations of P^s gives rise to a massless state corresponding to one of the 10-components. 
The fibration of these curves over Cio is what one calls the matter surfaces O^q, k = 1, ... 10. 

As an example of one these ten surfaces, consider the complete intersection within the 
ambient fivefold given by 



As discussed in |41j . C24 describes a P^ fibered over the curve Cio : {ai = 0} n on ^3, as IS 
manifest from the form of the resolved Tate constraint Pt (|4.22p : If 62 = 64 = ai = 0, the Tate 
constraint is satisfied so that the locus lies inside I4. Furthermore all are P^ fibered over 
, hence the surface is itself fibered over the curve Cio in the base. Note that this surface 




in (j4.24p and (|4.25p . Using these results one finds 





(4.29) 



O24 : {62 = 0} n {64 = 0} n {ai = 0}. 



(4.30) 
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can be written as a complete intersection of the ambient fivefold X5, but not as a complete 
intersection on the Calabi-Yau I4 itself. 

After this brief review we finally turn to the instanton wrapping the divisor Dm- We will 
show under a specific technical assumption that if D\,^ intersects the SU{5) divisor W'' one 
can construct surfaces lying within Dm that are analogous to the matter surfaces. However 
for generic choice of complex structure moduli these surfaces cannot be realised as algebraic 
surfaces. By smoothly deforming the complex structure moduli to the special locus where the 
matter surfaces become algebraic one can compute the integral ()4.14p of l*G4 over the matter 
surfaces within Dm and deduce a selection rule for the instanton. 

As an example we consider the locus 

C24|dm : {62 = 0} n {64 = 0} n {Dm = 0} (4.31) 

in the ambient fivefold, which is the obvious candidate for a surface analogous to C24 lying 
inside the instanton. Of course the following arguments immediately carry over if one starts 
with one of the other matter surfaces C^q rather than C24. If we assume for the moment that 

this surface can be taken to lie inside the instanton divisor Dm C I4, we obtain a selection 

(2 2) 

rule by integrating i*G4 over it. For the fluxes in H^er O^a) considered here this integral is a 
sum of terms, each of which can be written as a complete intersection of five divisors within 
the ambient fivefold 

I L*G4 = I E2AE4ADM A G4. (4.32) 



The integral J^,^^ i*Gi has been performed in [JT] and expressed as the integral of a fiux- 
dependent 2- form on Bj, over the locus Ciq. The result of (j4.32p is therefore the same integral 
over the curve D\.^ n W*^ in i?3 . We thus conclude 



t*G^ = - I D^^j AW^ AiGJC-bW""), I L*Gf = -[ dIjAW'^AJ^^. 



C24\dj^,j -J JB3 ■'C24\dj,j ^ JBj, 

(4.33) 

The value of the integral would be unchanged if we had taken one of the other matter surfaces 
C^Q instead of (gJO]). 

Let us now justify our assumption that the algebraic cycle (j4.3ip in X5 could be used to 
obtain a selection rule. Note that while this surface manifestly lies inside Dm C X^, for generic 
complex structure moduli it is not completely contained within {Dm = 0} n I4. We will argue 
that there is however an associated non-algebraic surface within {Dm = 0} n I4, over which 
we can integrate G4 with the same result as if we had integrated over ()4.3ip . Let us consider 
a special geometric setup for which there exist holomorphic polynomials a, j3 with associated 
effective divisor classes [a], and an integer n such that we can write 

(ai)"+i = aD\j + pW'^, (4.34) 

with ai in the class K,. Now we can consider an auxiliary fourfold defined by Pxt where Pt is 
given by ()4.22p with ai replaced by ai. The condition ()4.34p ensures that C24I _Dm entirely 
contained within the deformed fourfold {Pt}- As {Pt} is obtained from I4 by a smooth 
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deformation this shows that there is a non-holomorphic surface associated to C24\dm aheady 
contained in the original fourfold. In the course of this deformation the 4-forms dual to the 
surfaces picks up pieces in H^^'^^ while keeping a piece in H^'^''^\ What counts is that the value 
of the topological integral ()4.33p is unchanged under such a deformation, as G4 is of type (2,2). 
Therefore, even though the surfaces C^q are in general not realised as algebraic surfaces within 
Dm for generic complex structure moduli, the integral J^,k l*G4 takes the value ()4.33p . 

It remains to comment on how restrictive the assumption ()4.34p is. Note that if the instanton 
and GUT divisor are chosen in such a way that 

[DU < [ai], [W''] < [a,] (4.35) 

in the class-theoretic sense, then (j4.34p can obviously be fulfilled with n = 1. Now in most 
applications to model building one considers bases constructed as complete intersections in a 
toric ambient space and rigid instanton divisors D^j. At least in this toric setting it is clear 
that rigidity implies that the polynomial D^j should be chosen to have as low a homogeneous 
degree as possible, so that ()4.35p will often be fulfilleci^. To be slightly more general let us note 
that in many models K. will be a big divisor, i.e. it will lie in the interior of the pseudo-effective 
cone. This is certainly the case for all base spaces that are obtained from Fano surfaces by 
blow-up. In fact, to the best of our knowledge, all examples of F-theory GUT models presented 
so far in the literature are of this form. In this case it is always possible to find an integer n 
such that [a"] — [W^] and [a"] — [D^j] are effective. In other words, we can always find eff'ective 
classes a, f3 and 6 such that 

(ai)'' = aDlj + + 6. (4.36) 

Therefore the problem is reduced to the question of whether by a suitable deformation of the 
polynomials involved one can achieve — 6 = a". While we expect this to be generically 
possible due to the freedom of choosing suitable representatives for a, /3, and W^, it would 
be interesting to obtain a rigorous proof, and to see how this should be generalised in case K, is 
not a big divisor. 

Let us summarise the upshot of the considerations presented above. Although in general 
the matter surfaces will not lie inside the instanton as holomorpic surfaces, we expect that one 
can smoothly deform the geometry in such a way that holomorphicity becomes manifest. This 
implies that the matter surfaces were already present inside the instanton in the undeformed 
geometry, albeit without a holomorphic representative. In the course of such deformations the 
Poincare dual of the surface picks up pieces of Hodge type (1,3) and (3,1), but this does not 
change the value of the integral of the (2,2)-form G4. Hence we can safely evaluate the integrals 
()4.14p in the geometry ()4.34p . It would be interesting to understand how to generalise the proof 
to the general case. 

Our presentation has focused on the surfaces fibered over the matter curve Cio in the above, 
but similar arguments hold for the remaining matter surfaces. All these integrals must vanish 
in order for the instanton to contribute to the superpotential without insertion of additional 
M2-Brane vertex factors. Note that the number of actual selection rules will in general be much 
smaller than the number of surfaces that can be constructed in this way, as matter surfaces from 
different enhancement curves are in general not linearly independent and the integrals of G4 

^^In order to construct a consistent Weierstrass model also the class [W^] cannot be too large as the class 
[a2,i] =2K.~ [W^] must be effective. 
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over all matter surfaces of a given enhancement curve are the same. In the next subsection we 
will discuss the relation to the known selection rules in the Type IIB dual and we will confirm 
that the selection rule from U{l)x charge and from the Cio surface generate the entire set of 
selection rules for models with an orientifold dual. This is intuitive because the Cio surface is 
sensitive to charged zero modes between the instanton and the SU{5) brane stack only, while 
the U{l)x selection rules is sensitive also to modes between the instanton and the /i-locus of 
the discriminant, which is responsible for the U{l)x- 

Although we have considered specifically an SU{5) x U{1) model in the above, the general 
argumentation is independent of the specific F-theory model. We expect that the surfaces which 
contribute non-trivial selection rules via (j4.14p are surfaces which correspond to P^s fibered over 
curves of singularity enhancement in the base. 

It is interesting to compare to situations without an extra U{1) group, in this case to generic 
SU{5) Tate models. The surface —Dm A wx ceases to exist as an algebraic surface for generic 
complex structure moduli. What takes its place [HU] (see also [HlllS] in the SU (5) context) is 
a 4-cycle that can be written as a complete intersection only on the ambient fivefold X5, but 
not on I4. Its dual 4-form is an element of H'^^^{Yi). This surface now leads to a selection rule 
due to the Freed- Witten anomaly similar to the one for the matter surface^!. The resulting 
selection rules are expected to be equivalent in form to the ones in ()4.28|4.29p . even though no 
interpretation in terms of a ?7(1) charge is possible if the U{1) is higgsed. The relevant integrals 
can be worked out by adopting and extending the techniques in [39], but we do not present this 
computation here. 

4.3 Comparison with selection rules of Type IIB and U{1) charges 

Having discussed selection rules for the absence of chiral charged zero modes from an F-theory 
perspective, let us now see how this is related to the more familiar Type IIB picture. 

In Type II orientifolds the intersection of a D-brane instanton and a spacetime-filling D- 
brane hosts chiral charged zero modes in the sector of boundary changing open strings [3l[5l[6], 
which is straightforward to quantise. In the case at hand, the net number of charged modes 
between an instanton along the divisor De on a stack of Na D7-braned along the divisor 
Da is given by the chiral index 



This counts the chirality of states in representation (Nyi,!^;). As in section [3.11 we use the 
notation = 2na' F — B. Note that apart from the brane fiux Ta on Da also the instanton 
fiux Te contributes to this chiral index [9j. Furthermore the contribution of the continuous 
moduli drops out of the chiral index as they appear in both Ta and Te- The situation is 
different for the discrete i3+ moduli in the case of an 0(l)-instanton as the full orientifold-even 
part is projected out. Therefore the chiral index for an 0(l)-instanton depends on F'^ 
rather than Fa, matching the U{1) charge of the instanton as discussed below. 

Equivalently, the presence of a net number of chiral zero modes can be detected from the net 
charge of the instanton action under each U (1)a- For a stack of Na spacetime-filling D7-branes 

Note that in this recombined case also G4 is no longer in i/ier (i4). 




(4.37) 
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on a divisor and the image stack along D'^ this works as follows. The Poincare dual 2- 
forms are expanded as [Da] = ^C^Ua + ^C'^uja and [D'j^] = ^C^Ua — ^CX^a- The Stiickelberg 
mechanism then leads to a gauging of the shift symmetries of the scaiars c"" and Ca , such that 
under a gauge transformation A-^ — )• A^ + 4-dA"Xj of the diagonal U{1)a the chiral fields (I3.2p 
shift as 

27r 

Ta Ta- l^^A {^alB-) ^A + ^"^c J^A ^a) ■ (4.38) 

The partition function (|3.17p of the 0(1) E3-instanton with a fixed fiux configuration then also 
shifts as 

^-Se ^ ^-^gA^^^-SE ^ = ^^^^ c% C\ {T% + K^p-,C% . (4.39) 

Note that only the pullback instanton flux contributes to the instanton charge, not the variable 
flux . It is simple to check that the charge qA precisely returns the chiral index ()4.37p . 



As a consequence of the non-trivial J7(1)a charge qA associated with a given instanton 
flux, each conflguration with charge qA contributes now to a superpotential term of the form 
W ~ Oexp(— S^;), where O is an operator involving open string fields with \J{X)a charge —qA- 
The sum over the instanton fiuxes in the instanton partition function then splits into various 
contributions. Each such contribution involves the sum over the full allowed lattice of variable 
instanton fluxes J-^ together with the sublattice of allowed pullback fluxes whose induced charge 
matches that of the respective operator O. 

In order to compare the orientifold description with the results of the previous subsection we 
flrst recall from [42] the D7-brane and flux configuration of the Type IIB model corresponding 
to the F-theory setup with gauge group SU{5) x U{l)x- The IIB setup consists of a stack of 
5 D7-branes on a divisor Da on together with its orientifold image D'j^ and one additional 
D7 on Db = 4:Do7 — 2D a — 3-D^ along with the associated image stack. For the reader's 
convenience we reiterate that the F-theory base is the orientifold quotient of X3, i.e. there 
is a projection p : X3 — t- B3. The SU{5) divisor and the instanton divisor D\^ in are 
given as 

D+ = Da + D'a= p*>V^ De = P*D\,. (4.40) 

The class Dqj of the orientifold plane maps to the anti-canonical class of i?3, K, = p*Do7- 
Finally mind the factor of 2 in 

/ p*{D^)Ap*{D^)Ap*{Dt)=2 [ D^AD^ADl (4.41) 

The two Abelian gauge symmetries U{1)a and U{1)b on Da and Db are individually 
massive even in absence of gauge flux, but there is a massless linear combination 

U{l)x = ^{U{l)A-5Uil)B), (4.42) 

^''The explicit factor of has been included to account for the difference in normalisation between the gauge 

field A'^ in the type IIB limit and the gauge field A'^ appearing in the expansion of C3 in (|4.13[) . see [36]. This 
implies that the gauge parameter A appearing here can be identified with the parameter appearing in l|4.17p . 
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which only becomes massive if gauge flux is turned on. The Type IIB flux conflguration corre- 
sponding to the G'4-fluxes defined in ()4.24p and ()4.25p is given by [4^ 



Gf: H = h^^ ^i = -\^^ -^A,B = 0, (4.43) 



G^ /+ = |AZ)o7, -^^ = 0, -^1,5 = (4.44) 

with 

F = p*F^. (4.45) 

In particular we may compute the charge of the E3-instanton under the massless U{l)x of 
(SaD as 

q"" = \{^XA,E - ^Xb,e)- (4.46) 

The factor of 5 in front of XA,E is due to the fact that the zero modes at this intersection locus 
transform in the 5 of the U{b) along Da- 

Let us consider first the fiux configuration (j4.43p corresponding to . From the chiral 
indices 



XA,e{G^) = -^J^ DeAD+AF=-^J^ dIjAW'^AT'^, (4.47) 
XbMGa) = 7 / DeAD+AT=1[ D\jA{^iC- 5W^) A T"" (4.48) 

4 JX-, ^ JB:, 



1X3 ^ JB3 

one deduces the overall U{l)x charge 

1{5XA,E - 5xb,e) = -10 / A{iC- ^W^) A ^^ (4.49) 

This precisely reproduces the F-theory result (|4.28p . 

In both pictures the first necessary selection rule for generation of a superpotential without 
insertion of extra net charged matter is therefore that (j4.28p 

D\j a (-^ - -W^) A -F^ = 0. (4.50) 

From the type IIB perspective it is clear that the absence of chiral charged zero modes actually 
requires the vanishing of the chiral indices XA,E and XB,E individually, not just in the combina- 
tion ()4.46p . The complementary condition can be obtained in IIB by demanding the vanishing 
of the net chiral index with respect to any other U{1) linearly independent from ()4.42p . However 
all other combinations of U{1)a and U{1)b are massive, and therefore not directly visible in the 
F-theory model [36j. Therefore the corresponding selection rule in F-theory cannot be derived 
in the manner presented above for U{i)x- Instead, it must be contained in the additional 
selection rules discussed in the previous subsection which were obtained by integrating G4 over 
one of the matter surfaces. Indeed, for the flux the selection rule (|4.33p just corresponds 
to the vanishing of XA,E- Together with (|4.50p this suffices to guarantee the vanishing of each 
chiral index. 
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Put differently, the massive U{l)s, even though inivisible in the spectrum of the F-theory 
compactification, do remain as selection rules in F/M-theory. They enter through the Freed- 
Witten anomaly, which enforces that l*G4 = on the M5-instanton. While the spacetime 
interpretation is different, the effect is the same in F-theory compared to Type IIB orientifolds. 

That the massive U{l)s would continue to play a role as selection rules was conjectured 
from a different perspective in [36]. Formally, the massive U{l)s can also be made visible by 
including a set of non-harmonic 2-forms and 3-forms in the dimensional reduction of the M- 
theory supergravity. In this approach the gauging induced by the massive U{1) is encoded in 
the intersection form of these non-harmonic forms, and a definition of the instanton charge 
along the lines of ()4.19p is possible by replacing wx with the non-harmonic 2-form associated 
with the massive U{1). The results of the present analysis are in agreement with |36j even 
though no use of the non-harmonic forms has been made. 

For the fiux configuration the charged zero mode indices follow as 

Xa,e{GI) = -^I Do7ADeAD+ = -'^ [ JCAD^AW, Xb,e{G^4) = 0- (4-51) 

The resulting U{l)x charge 



-A / ICADIjAW (4.52) 

is in agreement with the F-theory result ()4.29p . Since Xb,e{G^) = 0, there is no further selection 
rule in Type IIB. At first sight this may seem puzzling, as in the F-theory setup we obtained 
the two selection rules ()4.29p and ()4.33p . To understand this seeming discrepancy, we must 
recall that a smooth Type IIB limit actually exists only if the conifold point |42y78j 

Da n {ai = 0} n {03,1 = 0} (4.53) 

is absent, with the polynomials ai and 02,1 defined in ()4.2ip . Moreover we have assumed from 
the beginning that the SU{5) divisor is smooth. Type IIB orientifolds corresponding to 
this class of models satisfy the relation 



{D+f - {D-^f = 2D\Do7. (4.54) 

Expression (j4.33p therefore equals 

\ r _ 2A I' A 

- / DliAW'^A (6£ - 5W^) = -— / DeAD\aDo7-i:DeA (D^f. (4.55) 
JBs t) Jxs ^ 



Next note that for a general divisor the intersection H D'^ splits into a component along 
the 07-plane and a component away from it. Cohomologically the latter is proportional to 

A -D^loff 07 + A — A D^. In a Type IIB orientifold corresponding to an F-theory 
model with smooth SU{5) divisor W^, the component away from the 07-plane is absent because 
this is the locus that would translate into self-intersections of W*^. Now, Jj^^ De AD^AD^ = 
because De is orientifold even; furthermore we argued that the intersection D^j n can be 
deformed inside oi = 0, i.e. the integral fj^^ De A D\ A D\\oE07 = 0. In conclusion the last 
term in (|4.55p vanishes. 
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Hence as long as there exists a smooth Type IIB Hmit we indeed obtain only one selection 
rule for this type of flux configuration, as expected. Note however that (|4.53p is not required 
to be absent in a general F-theory model, and the number of independent selection rules may 
be higher in this case. In fact, the locus ()4.53p describes a point of enhancement to Eq [78] . 
and one might speculate that new charged zero mode states can appear at this exceptional 
singularity. This intuition is supported by our observation that additional selection rules not 
encountered in Type IIB can appear in the presence of a point of exceptional enhancement 
inside the instanton divisor, but more work is needed to understand this microscopically. 

We would like to close this section with an even more speculative remark: For fluxed D3- 
instantons in Type IIB orientifolds, the chiral index (|4.37p counting the charged zero modes 
depends not only of the 7-brane fluxes, but also on the instanton flux J^e- In view of (|4.39p 
this effect induces an instanton flux contribution to the net U{l)x charge of the instanton and 
can in particular be used to cancel the chirality induced by D7-brane gauge flux |9j. On the 
other hand, no analogous phenomenon has been observed for the M5-instanton flux quanta T-L 
in the present treatment. A possible resolution of this mismatch is related to the fact that the 
relevant D3-instanton fluxes arise from 2-forms in H^^{De) with non-vanishing pullback onto 
the D7-brane stacks. As indicated in section 13.21 the uplift of such 2-forms to the cohomology 
of Dm is unclear. Formally it is possible introduce a set of non-harmonic 3-forms on Dm to 
capture the missing H fluxes such as to recover the U{1) charges, in a similar spirit to the 
analysis in [30]. It would be desirable to understand, however, whether a different and more 
concrete description of this effect is possible. We hope to return to this interesting point in the 
future. 

5 Conclusions and open questions 

In this paper we have studied the superpotential due to M5-instantons in F-theory compactifi- 
cations. The discussions are framed in a general F-theoretic setting, but are augmented, where 
possible, by comparison with known Type IIB results. 

In the first part of the paper we have discussed the classical partition function of M5- 
instantons, focussing on instantons wrapped on vertical divisors of the elliptically fibered four- 
fold of F-theory. These can be related to E3-instantons in the corresponding Type IIB model. 
By comparing the results with the analogous expressions for the E3-instanton we have been 
able to identify the correct partition function of the M5-instanton, which amounts to choosing 
a specific line bundle on the intermediate Jacobian of the M5-instanton divisor. Our result 
shows that there is a canonical choice independent of the precise geometry of this divisor. The 
calculations required to extract the correct line bundle directly in M-theory are highly non- 
trivial, and it is not obvious that such a simple canonical choice should exist. The M5 partition 
function identified using the duality with Type IIB is in agreement with the results of [16H19j . 
which calculate the partition function on a flat torus resp. on KSxT"^. However, our result 
is applicable to a much larger class of instanton geometries; in particular the existence of a 
smooth Type IIB limit only poses constraints on the form of the discriminant locus, but not of 
the M5 divisor. Thus we expect this result to hold also if no such limit can be taken smoothly. 

In the second part of this work we have focused on the question of how the presence of G4-flux 
restricting non-trivially to the instanton affects the M5-instanton superpotential contribution. 
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Our discussion is applicable to general G4-fluxes and valid even in absence of a smooth Type 
IIB limit. It is well-known that in Type IIB gauge fluxes can generate a chiral spectrum of 
charged fermionic zero modes on the E3-instanton. The presence of such chiral zero modes 
implies that the instanton can at most contribute to superpotential couplings involving charged 
open string fields. We have discussed the corresponding effect from the perspective of M-theory, 
where it has a close relationship with the Freed- Witten anomaly cancellation condition [32p38j . 
This condition implies that a consistent theory of the M5-instanton in the presence of G4-flux 
requires the presence of a suitable configuration of M2-branes ending on the instanton. We 
have argued that in the resulting superpotential contributions, these M2-branes play the role 
of the charged open string fields in the dual Type IIB picture. Related recent work on these 
instanton zero modes can be found e.g. in [2n[22l[32| [38] . 

In many applications one is interested in generating superpotential contributions that do 
not involve charged operators. A necessary condition for this is encoded in the selection rule 
L*G4 = 0. This selection rule guarantees absence of chiral charged fermionic zero modes. Of 
course there can exist further neutral zero modes which forbid the generation of a superpotential 
term. These neutral zero modes have been discussed e.g. in [211[231 - l26] and are not considered 
further in this work. Focussing on a specific set of non-Cartan G4-fluxes that were recently 
constructed in |40H43j . we have discussed a set of explicit 4-cycles on the M5-instanton that 
can be used to numerically test for the condition l*G4 = by evaluating the integral of G4 
over these cycles. Following this discussion in a general F-theoretic setting, we then proceed to 
illustrate these ideas in the context of the explicit SU{5) x U{1) model as considered in [42]. In 
particular we have checked that if a smooth Type IIB limit exists, the selection rules obtained 
in this manner correspond exactly to the vanishing of the chiral indices counting zero modes at 
the intersections between the E3-instanton and D7-brane stacks. 

It is well known that in the Type IIB setting these selection rules for the absence of chiral 
charged zero modes can be obtained by considering the charge of the instanton under Abelian 
gauge symmetries present in the model. As is to be expected, we have found that a correspond- 
ing observation can be made in F-theory. Namely, in models including a massless U{1) gauge 
symmetry the integral of G4 over some of the above 4-cycles computes the instanton charge 
under this massless U{1) symmetry. Furthermore, we have shown that the additional selection 
rules which arise from the remaining types of surfaces can be interpreted as instanton charges 
under massive U{1) gauge symmetries, confirming the expectations of [36J. 

There are several obvious directions for future study. We have performed the explicit match 
of the partition functions for the case of 0(1) E3-instantons wrapped on a rigid, irreducible 
divisor invariant under the orientifold action. A number of questions arise when one relaxes 
these conditions. If one considers a non-rigid divisor one obtains in Type IIB a non-trivial 
F-term condition restricting the instanton flux which can contribute to the partition function. 
In the dual F-theory setting one can immediately identify a corresponding set of M5-instanton 
fluxes which are expected to be ruled out by a similar condition. However, the obvious uplift 
to F-theory of the Type IIB F-term expression does not seem sufficient to rule out all of these 
fluxes, and it would be interesting to study how the full required constraint arises in F-theory. 
Another interesting question would be how to describe the uplift of Type IIB U{1) instantons, 
which admit fluxes of even orientifold parity contributing to the partition function. Such fluxes 
uplift to 2-forms on the M5-instanton with no obvious interpretation as M5 3- form flux T-L. 

Further points requiring further investigation concern the charged zero modes of the M5- 
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instanton and the resulting selection rules. While we have discussed in general terms how the 
surfaces in the M5-instanton which can be used to detect non-trivial selection rules should 
arise, the construction of these surfaces in the framework of the model considered in section 
I4.2.2l relied on a specific technical assumption. While it is clear that the technical assumption in 
question is not a necessary condition for the existence of the surfaces, it would be interesting to 
see how the proof can be generalised. In addition, it would be very nice to understand in more 
detail how the charged zero modes are to be understood in terms of M2-branes ending on the 
instanton and in particular how the appearance of these zero modes is related to the geometry 
of the singular elliptic fiber over an intersection with a D7-brane stack. A precise understanding 
of the necessary geometric requirements to give rise to such M2-brane states should also allow 
one to understand the role of vectorlike pairs of zero modes. Another approach would be to 
use more extensively the correspondence between M2s ending on the M5-instanton and non- 
trivial T-L-Qux configurations accompanied by a suitable deformation of the M5 in the M2-brane 
directions. This would allow one to extract information on the zero modes by studying instead 
the zero modes of this "H-field configuration. 

Finally, it would be interesting to consider how non-zero Ti-Hux affects the instanton selec- 
tion rules. In Type IIB theory it is known that instanton fluxes which restrict non-trivially to 
the intersection of instanton and brane contribute to the chiral index or equivalently the U{1) 
charge of the instanton [S]. However, the uplift of precisely this type of fluxes to F-theory is 
problematic due to the degeneration of the elliptic fiber over the location of the D7-branes. 
In [36J it was conjectured that 2-forms with negative orientifold parity restricting to the D7- 
branes uplift to non-harmonic 3-forms in F-theory. Following this idea suggests that IIB in- 
stanton flux that contributes to the chiral index should be described by non-harmonic 'H-flux 
on the M5-instanton. Intriguingly, one finds exact match with the Type IIB expression if one 
computes the U{1) charge of the M5-instanton after including ?^-flux along the non-harmonic 
forms introduced in [36j. However, it is so far unclear how the appearance of non-harmonic 
"H-flux is to be understood from a purely M-theoretic perspective. We look forward to revisiting 
these and related question in the future. 
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A M-theory reduction in the democratic formulation 

The low-energy limit of M-theory is conventionally described by the action 

5 = ^1 d^'xV^R - i / (g^A *G4 + ^Cs A G4 A G^ (A.l) 



35 



in conventions where nf^ = 1. The dimensional reduction to three dimensions of the mass- 
less harmonic sector of this action on a Calabi-Yau fourfold as well as its F-theory limit has 
been extensively studied e.g. in [35l[79l[80] . Furthermore in |36j it was discussed how certain 
non-harmonic forms can be included in the dimensional reduction to describe massive U{1) 
symmetries. However, in order to describe the interaction between M5-instantons and the C3 
field it is advantageous to stray from this conventional path and use instead a democratic for- 
mulation which includes also the magnetic dual Cq of C3. This mirrors the situation in Type 
II string theory, where one uses the democratic formulation upon including D-branes. 

The duality-symmetric eleven-dimensional supergravity theory has been considered by the 
authors of [81], who find that one needs to introduce an additional auxiliary scalar field to 
obtain a manifestly covariant theory. In the following we will not follow this approach; instead 
we will describe the theory by means of a simpler pseudo-action which is supplemented by a 
set of duality constraints to be imposed at the level of the equations of motion. This is again 
analogous to the approach conventionally followed in the setting of Type II string theory. 

In the theory defined by (|A.ip the dynamics of C3 are determined by the equation of motion 

d*G4 + ^G4 AG4 = (A.2) 

as well as the Bianchi identity (iG4 = 0. In close analogy to the democratic formulation of Type 
II string theory one can define a dual field strength 

G7 = dCQ + ^C3 A G4 (A.3) 

and consider the action 

SD = \j d'^x^R - 1 y (^G4 A *G4 + ^G-r A *Gr^ . (A.4) 

Supplementing this pseudo-action with the duality relation 

* G7 = G4 ^ G7 = - * G4 (A.5) 

one easily checks that it reproduces precisely the original equations of motion ()A.2p as well 
as the relevant Bianchi identities. The prefactor of the second term can be fixed by requiring 
that integrating out Cg by introducing a Lagrangian multiplier term — J dC^ A dC'i leads back 
to the original action (|A.ip . While the form of the action (|A.4p is well known, to the best of 
our knowledge its dimensional reduction to three dimensions has not been performed in the 
literature. This will be the subject of this appendix. 

The advantage of using the democratic formulation when working with M5-instantons is that 
it explicitly involves the field Cq which couples directly to the instanton world-volume. This 
coupling involves a set of scalars Cq which as we will see below become part of the complexified 
Kahler moduli. Performing the dimensional reduction of ()A.4p in the democratic formulation 
allows us to determine the correct normalisation of the Kahler moduli. This in particular allows 
us to compute the dependence of the M5-instanton action on the Kahler moduli including all 
prefactors and to perform the precise match with the Type IIB expression. 

It is worth noting that the dimensional reduction of the usual action (jA.ip yields a three- 
dimensional theory involving vectors which can in three dimensions be dualised to scalars 
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Cq |79y8U). However even if one does this and rewrites the bulk action in terms of these scalars 
Cq,, one does not know precisely how the so obtained Cq, should appear in the instanton action. 
In fact as we will see there is a non-trivial shift between the fields appearing naturally in the M5 
action and the fields that form part of the Kahler coordinates, which one can only determine 
precisely when performing the reduction in the democratic formulation. 

To carry out the dimensional reduction of the action (|A.4p let us first focus only on the mass- 
less fields described by harmonic forms and introduce the bases {i^a} resp. {ui^} of H^'^{Y4) 
resp. H^'^iY^), {r/™} H^iY^) and {aa,/3^} resp. {a^Jb} of H^iY^) resp. H^iY^). These 
forms are chosen to obey 

coAACj^ = ^5f, A,J: = l,...,h^'\Yi), (A.6) 
r,™Ar/" = 5™", m,n = l,...,h\Y4), (A.7) 



a,b = l,...,h''{Yi). (A.8) 

JaaAl3b = J/3^Aab = j 

Recah that H^'^iY^) can be split into several distinct subspaces, namely the forms oja which 
constitute the uplift of ff^'^(i?3), the form cjq which is Poincare dual to the base as well as 
a set of resolution divisors 00 j describing massless U{1) gauge symmetric^. To simplify the 
notation let us further assume that h^''^{B3) = 0, so that we have the intersection numbers [36] 



A pb ilfCaacS''^ A = a 

= (A.9) 

WA Aoa Aafc = y WA A/5'' A/3* = 0, (A. 10) 

ujA A ujj: A Lvn A uie = ^/Caehb- (A. 11) 

Recall that in the case where the model admits a Type IIB limit on the double cover of 
i?3 the Ua and /?* are the uplifts of H^^{X3). In this case the intersection numbers JCaab 
and JCa/^'Y = can also be identified with the corresponding intersection numbers on X3. 

However we reiterate that the existence of a smooth orientifold limit is of course not necessary 
for this reduction. 

For further convenience let us rewrite the intersection numbers above in terms of the fol- 
lowing equivalent set of equations in cohomology 



(A.12) 



We may now expand C3 and Cq into the bases of the cohomology groups defined above as 



WA A A wn 




Oa A/3* 




aa A UJa 


= —K^aac^'^'^Pb, 


f3^AuJa 


— JC 


aa A ab 


= 0= /3"A/3*, 


aa A UJA 


= 0= fi^AuJA, A = 0, /, 


OJA A 




=^ LJA Ar]"^ 


= Ib'^'^RnAllUJ^- 



^'^We do not need to distinguish at this point between forms describing (7(l)s in the Cartan of a non-Abehan 
gauge group and additional Abehan gauge group factors. In this appendix Y4, is always taken to denote the 
smooth, resolved fourfold. 
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follows 

C3 = ^UK + d'aa + hP\ (A.13) 

Cq = CAcd^ + tJaAa" + A^b + rm^v'^- (A.14) 

Here c", bh, ca are scalars, while A^, Ua, are vectors and the are spacetime 2-forms, 
which are non-dynamical in three dimensions. We also include non-trivial G4-flux 

G4 = dC73 + J-„r/"^, (A.15) 

which is dual to the 2-forms r^- We are now in a position to insert these expansions into the 
action (|A.4p and integrate out the internal space I4. To simplify the resulting expression let us 
finally introduce the notation {pi} = {/O*} = {a",/?;,} and 

A^AS := y WA A *ujj: ^ j ^ ^ i^"^^' (A.16) 
Mij := J Pi A *p, ^ J p'a *~p> = ^M'^, (A.17) 
7?"A*r/'", (A.18) 



where N^^ denotes the inverse matrix of Aas etc.. Ignoring the Einstein-Hilbert term in ()A.4p 
for the moment one finds the following action for the fields arising from C3 and Cg 

=-|/ffii,2[ Naj^F^ A*F^ + Mijdc' A*dc^ + ^N^^VcaA*Vcj: (A.19) 

+ lM'^VUi A *VU, + Jp'^'Pr^ A *Pr, + P™"-F„^J; * 1 
o 2 

The shifted fields and covariant derivatives appearing here are given by 

CA, 



VcA := dcA + h@A^A^ + ' °' 



Ua - ^ICaacS'XA'^, Ui = Ua (A.20) 



ICaacS'^XF'^, 



A = 


--0, I 


A 


= a 


A = 


= 0, I 


A 


= a 


U 


= Ua 


u 


= yfc' 


Ui 


= Ua 


Ui 


= F6' 



u 

VUi := dUi+ , ,^ ^ , 

Vrm := drm + ^RmAJ^A^ A . 
The flux-dependent quantity ©ae controlling the gauging of the scalars ca is given by 

Oas = 2J-„(^"^"i?„AS =2 y a;A A A G4. (A.21) 

The next step is to eliminate the redundant degrees of freedom using the duality relation 
(jA.Sp , which after dimensional reduction can be written in terms of the three-dimensional fields 
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2Mij * dcP = -VUi ^ *VUi = 2Mijdc>, (A. 22) 

2iVAS = -VcA ^ *VcA = 2iVAs * F^, (A. 23) 

-(5„„P"PJp * 1 = Pr^ ^ J-^ = * Prp. (A.24) 

However, when computing the equations of motion of the fields arising from ()A.19p one 
finds that they are inconsistent with the duality relation given above. This phenomenon is 
a result of using the pseudo-action (|A.4p instead of the fully-fledged covariant action of [81] 
(see also [82]). It is also familiar from reductions of democratic Type II supergravity, compare 
e.g. |5 HI83f[H5] for discussions in the Type II setting. In order to restore consistency one must 
shift the deflnition of the field strengths Pca to 

After adding a Lagrangian multiplier term ^ f 6"^'"'J-ndrfn to the action one can treat drm as an 
independent field and finds that its e.o.m. precisely give the duality relation ()A.24p . Integrating 
out dvm using these e.o.m. leads to 

=-Hmi,2[ Naj:F^ A*F^ + Mijdd A*dc' + In^^VcaA*Vcj^ (A.26) 
L 8 

+lM'^VUi A *VU, + Iqaj^A^ AF^ + Ip^^Frr^Fn * 1 
8 4 2 

Next one could eliminate either the vectors A^ or the scalars ca in favor of their dual fields 
leading to two different but equivalent formulations of the low-energy theory. As we want to 
study M5-instantons on vertical divisors of the form [Dm] = C^uJa we would like to keep the 
scalars Cq, which enter the instanton action via the coupling Sm5 ^ 27rz J^^^ Cg. On the other 
hand we would like to keep A^ resp. A^ in the theory to facilitate the match with the Type 
IIB theory, where they correspond to the off-diagonal components of the 4d metric resp. the 
massless U{1) gauge symmetries ^35j. This turns out to be possible if we assume that 

@a0 = 0, (A.27) 

such that Vca does not involve the dual degrees of freedom A". In fact, Qa/B oc f LOaAuJi^AG^ = 
is anyway required in order to obtain a consistent F-theory limit |35tl36j. Letting the indices 
r, s, ... run over the values 0, 1 here and in the following we again add a Lagrangian multiplier 
g f ^dcrAF^ to the action. From the definition ()A.16p and the different origins of the forms oja, 
ojr one infers that the submatrices Nf^i-j := (Na^) and A''(2) ■= i^rs) are separately invertible 
and positive definite. This implies that the same statement holds for their respective Schur 
complements 5(j), i = 1, 2, which are the inverses of the corresponding subblocks of the inverse 
matrix N^^, i.e. 5(i)„A^^p = 6?, S^2)apNP^ = 5l. Therefore the e.o.m. 

pr ^ 1 ^rA ^ ^^ 28) 



^^Here we used the fact that due to the block-diagonal form of the metric one has *ii (ojAr;) = ( — 1)'''' '''(*3a;)A 
(*y-^?7) for a p-form uj on R^'^ and a q-form 77 on Yi; this was also used in deriving (|A.19p . 
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obtained by treating dcr as an independent field suffices to completely determine T>Cs in terms 
of and Vca and can therefore be used to integrate it out. 

A similar argument shows that one can consistently eliminate and Ui after adding 
corresponding Lagrangian multiplier terms — |/ F'^ A dca — \dUi A dd . The final result of 
this dualisation can be concisely written in terms of the inverse of A^(i) and the Schur 

complement 5(i)rs = -^rs — NraN^i^N^s and readJ^ 



Sc =-i/Mi,2f S^i)rsF' A*F'-N^^^NprVco,AF' + -N"^VcaA*Vcp (A.29) 



1 

+Mijdc' A *dc^ + lersA^ AF' + P^-Fr^j; * 1 



The vectors and scalars combine with scalars arising in the expansion of the Kahler 
form J = v^ujA into three-dimensional vector and scalar multiplets. These scalars enter the 
effective action through the reduction of the curvature scalar, which was performed in [79] . 
After combining the result with (jA.29p and performing a Weyl rescaling g^i, — )• V~^g^u with V 
the volume of Y4 one obtains 



I [ d^x^R - - f [V5(i),,(dr A *de + F^A *F') (A.30) 



+4GMNdz^^ A *dz^ + ^Mijdd A *dc> + ^G,,^'- A + 1^P^^T„,T^ * 1 



Here we have defined = and 



Va = 5,aV = I uaAJ^. (A.31) 



3! 



Although they will not be needed in the following the h^'^^iY/^ complex structure moduli 
are included for completeness. Given a basis {^m} of H^'^{Y4) they are defined by Sg-fj 



M 



Hi] 



J_^^fc/r?i^(j)^^^^^^^^ while the metric is given by 



__/^mA$^ , . 

/f^A^ • ^^-^^^ 

The action (|A.30p can be put into the general form of a gauged three-dimensional N = 2 
supergravity as given in [36]. To do this one combines the scalars and hb into complex scalar 
fields A^" = — if°''^bb with a suitable function of the complex structure moduli [36] 179] 
and defines the complexified Kahler moduli 

ta = 2Va + ica = ■^^K-aKT.nv^v^v^ + ic„. (A.33) 

If one performs the F-theory limit of vanishing fiber volume e as in one finds that the fields 
ta scale at zeroth order in e as 

ta — > -ICap-rV^v'^ + iCa, (A. 34) 

Here we used the matrix inversion lemma which impUes A^""" = N^f^ + N°"' Si^i^rsN"^ . 
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where are the Kahler moduH of the base S3 and fCap-y = ^-^e the corresponding 

intersection numbers. If a Type IIB hmit exists, comparing with the usual expressions shows 
in particular that the fields can be identified with the corresponding axions appearing in the 
Type IIB Kahler moduli T^. 

Finally let us consider the modifications that arise upon inclusion of the non-closed forms 
wqa introduced in [36] to describe geometrically massive U(l) gauge fields. One change is that 
the indices r, s,... introduced above now run also over these additional values. To stick to 
the notation of [36] we introduce one such form wqa for each stack A of 7-branes and let the 
index i in r, s,... = 0, z,OA now run only over all Cartan C/(l)s. Note that although the wqa 
are individually non-closed and describe the massive diagonal U{1) on each stack of D7-branes, 
certain linear combinations may remain massless, as is the case in the SU{5) x U{1) model 
discussed in section! 



The non-closed forms induce a geometric gauging of the shift symmetries of the scalars c° 
resulting in the appearance of covariant derivatives 

Taking into account |36] 



[ 0, otherwise 

one also finds an additional piece in the covariant derivarives 

PC„ := dCa + ea^A"" - ICaacS^'c^dh " ^JCaacS'XNAClA"^^ . (A.37) 

This implies that the shift symmetries of the scalars c", Cq are gauged, such that under a gauge 
transformation A'^^ — )■ A^^ + dA^ of the diagonal U{1) the fields shift as 

Vc" = NaC%A^, S^ACa = ^ICaacS'^XNAC^A^ - O^oaA^. (A.38) 

This implies that the shifted field Ca is gauged only in the presence of fluxes as S/t^ACa = 
—QaOAA^, as was to be expected from the Type IIB perspective. 

Note that even if one picks fixed harmonic representatives for the basis forms above the 
equations ()A.12p still only hold in cohomology, as the wedge product of two harmonic forms 
is not necessarily harmonic. Upon inclusion of the non-closed wqa the additional exact pieces 
implicit in (|A.12p can lead to additional terms in the low-energy action. It is a non-trivial 
assumption that the intersection properties of the wqa are such that no such additional terms 
arise. 
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